UNIT IV

ADVANCED TESTING CONCEPTS

Performance Testing: Load Testing, Stress Testing, Volume Testing, Fail-Over Testing,
Recovery Testing, Configuration Testing, Compatibility Testing, Usability Testing, Testing
the Documentation, Security testing, Testing in the Agile Environment, Testing Web and
Mobile Applications.
4.1 PERFORMANCE TESTING
Performance Testing is a type of software testing that ensures software applications
perform properly under their expected workload. Performance Testing is the process of
analyzing the quality and capability of a product. It is a testing method performed to
determine the system’s performance in terms of speed, reliability, scalability and stability
under varying workloads. Performance testing is also known as Perf Testing.
Eg : During peak shopping seasons, such as “New Year”,”Diwali”, thousands of users may be
trying to access the site simultaneously. Performance testing can simulate this scenario by
creating virtual users to mimic real-life usage patterns

Performance Testing Attributes:

• Speed: It determines whether the software product responds rapidly.
• Scalability: It determines the amount of load the software product can handle at a
time.
• Stability: It determines whether the software product is stable in case of varying
workloads.
• Reliability: It determines whether the software product is secure or not.

Objectives of Performance Testing (Goals / Advantages):

• It uncovers what needs to be improved before the product is launched in the market.
• It helps to make software stable and reliable.
• It helps to evaluate the performance and scalability of a system or application under
various loads and conditions.
• It helps to identify bottlenecks, measure system performance, and ensure that the
system can handle the expected number of users or transactions.
• It also helps to ensure that the system can handle the expected load in a production
environment.

Some important types of Performance Testing:

1. Load testing: It checks the product’s ability to perform under anticipated user loads. The
objective is to identify performance congestion before the software product is launched in
the market.
2. Stress testing: It involves testing a product under extreme workloads to see whether it
handles high traffic or not. The objective is to identify the breaking point of a software
product.
3. Volume testing: In volume testing, large number of data is saved in a database and the
overall software system’s behaviour is observed. The objective is to check the product’s
performanceunder varying database volumes.
4. Scalabilty Testing : In Scalability testing , the software application’s effectiveness is
determined by scaling up to support an increase in user load

Panimalar Engineering College Chennai City Campus 1

 How to conduct Performance Testing ?

Disadvantages of Performance testing :

 High costs
 Difficulty in simulating real-world scenarios
 Need for specialized knowledge and tools

4.2 LOAD TESTING

Load testing determines the behavior of the application when multiple users use it at
the same time. It is the response of the system measured under varying load conditions.
• The load testing is carried out for normal and extreme load conditions.
• Load testing is a type of performance testing that simulates a real-world load on a
system or application to see how it performs under stress.
• The goal of load testing is to identify bottlenecks and determine the maximum number
of users or transactions the system can handle.
• It is an important aspect of software testing as it helps ensure that the system can handle
the expected usage levels and identify any potential issues before the system is
deployed to production.
During load testing, various scenarios are simulated to test the system’s behavior under
different load conditions. This can include simulating a high number of concurrent users,
simulating numerous requests, and simulating heavy network traffic. The system’s
performance is then measured and analyzed to identify any bottlenecks or issues that may
occur.
Some examples of load testing :
 Users trying to download large number of files.
 Server that is running multiple applications
Some Load Testing Techniques:
1. Spike testing: Testing the system’s ability to handle sudden spikes in traffic.
2. Soak testing: Testing the system’s ability to handle a sustained load over a prolonged
period of time.
Objectives of Load Testing:
 Evaluation of Scalability: Assess the system’s ability to handle growing user and
transaction demands. Find the point at which the system begins to function badly.
 Planning for Capacity: Describe the system’s ability to accommodate anticipated
future increases in the number of users, transactions and volume of data.
 Determine bottlenecks: Identify and localize bottlenecks in the application or
infrastructure’s performance. Finding the places where the system’s performance can
suffer under load is partof this.
 Analysis of Response Time: For crucial transactions and user interactions, tracking
and evaluating response times.
 Finding Memory Leaks: Find and fix memory leaks that may eventually cause a
decline in performance.

Panimalar Engineering College Chennai City Campus 2

 Load Testing Process:
1. Test Environment Setup: Firstly create a dedicated test environment setup for
performing the load testing. It ensures that testing would be done in a proper way.
2. Load Test Scenario: In second step load test scenarios are created. Then load testing
transactions are determined for an application and data is prepared for each transaction.
3. Test Scenario Execution: Load test scenarios that were created in previous step are
now executed. Different measurements and metrices are gathered to collect the
information.
4. Test Result Analysis: Results of the testing performed is analyzed and various
recommendations are made.
5. Re-test: If the test is failed then the test is performed again in order to get the result in
correct way.

Metrics or parameters of Load Testing:

Metrics are used in knowing the performance of load testing under different circumstances. It
tells how accurately the load testing is working under different test cases. It is usually carried
out after the preparation of load test scripts/cases. Some of them are listed below.
1. Average Response Time
It tells the average time taken to respond to the request generated by the clients or
customers or users. It also shows the speed of the application depending upon the time taken
to respond to the all requests generated.
2. Error Rate
The Error Rate is mentioned in terms of percentage denotes the number of errors occurred
during the requests to the total number of requests..
3. Throughput
This metric is used in knowing the range of bandwidth consumed during tests. It is also
used in knowing the amount of data which is being used for checking the request that flows
between the user server and application main server. It is measured in kilobytes per second.
4. Requests Per Second
It tells that how many requests are being generated to the application server per second. The
requests could be anything like requesting of images, documents, web pages, articles or any
other resources.
5. Concurrent Users
This metric is used to take the count of the users who are actively present at the particular
time. It just keeps track of count those who are visiting the application at any time without
raising any request in the application..
6. Peak Response Time
Peak Response Time measures the time taken to handle the request. It also helps in finding
the duration of the peak time(longest time) at which the request and response cycle is handled
and finding that which resource is taking longer time to respond the request.

Panimalar Engineering College Chennai City Campus 3

 Load Testing Tools:

 WebLoad: It is a performance testing tool designed to simulate user load on web

applications and measure their behavior under various conditions
 NeoLoad: It is a performance testing tool used to simulate user traffic and measure
how well applications handle load and stress
 LoadNinja: It is a cloud-based performance testing tool that enables users to simulate
real-world user loads on their applications

Advantages of Load Testing

1. Identifying bottlenecks: Load testing helps identify bottlenecks in the system such as slow
database queries, insufficient memory, or network congestion.
2. Improved scalability: By identifying the system’s maximum capacity, load testing helps
ensure that the system can handle an increasing number of users or transactions over time.
3. Improved reliability: Load testing helps identify any potential issues that may occur under
heavy load conditions, such as increased error rates or slow response times.
4. Reduced risk: By identifying potential issues before deployment, load testing helps reduce
the risk of system failure or poor performance in production.
5. Cost-effective: Load testing is more cost-effective than fixing problems that occur in
production.
6. Improved user experience: By identifying and addressing bottlenecks, load testing helps
ensure that users have a positive experience when using the system.

Disadvantages of Load Testing

1. Resource-intensive: Load testing can be resource-intensive, requiring significant hardware
and software resources to simulate a large number of users or transactions.
2. Complexity: Load testing can be comp
3. lex, requiring specialized knowledge and expertise to set up and execute effectively.
3. Limited testing scope: Load testing is focused on the performance of the system under
stress, and it may not be able to identify all types of issues or bugs.
4. Inaccurate results: If the load test scenarios do not accurately simulate real-world usage, the
results of the test may not be accurate.
5. Difficulty in simulating real-world usage: It’s difficult to simulate real-world usage, and
it’s hard to predict how users will interact with the system.

4.3 STRESS TESTING

Stress Testing is a software testing technique that determines the robustness of software
by testing beyond the limits of normal operation. Stress testing is particularly important for
critical software but is used for all types of software. Stress testing emphasizes robustness,
availability, and error handling under a heavy load rather than what is correct behavior under
normal situations.
Stress testing is defined as a type of software testing that verifies the stability and
reliability of the system. This test particularly determines the system on its robustness and error
handling under extremely heavy load conditions. It even tests beyond the normal operating
point and analyses how the system works under extreme conditions. Stress testing is performed
to ensure that the system would not crash under crunch situations. Stress testing is also known
as Endurance Testing or Torture Testing.

Need (Purpose / Goal ) for Stress Testing:

• To accommodate the sudden surges in traffic: It is important to perform stress testing
to accommodate abnormal traffic spikes. For example, when there is a sale
announcement on the e-commerce website there is a sudden increase in traffic. Failure

Panimalar Engineering College Chennai City Campus 4

 to accommodate such needs may lead to a loss of revenue and reputation.
• To Display error messages in stress conditions: Stress testing is important to check
whether the system is capable to display appropriate error messages when the system
is under stress conditions.
• To check if the system works under abnormal conditions: Stress testing checks
whether the system can continue to function in abnormal conditions.
• To Analyze the behavior of the application after failure: The purpose of stress
testing is to analyze the behavior of the application after failure and the software should
displaythe appropriate error messages while it is under extreme conditions.
• To Uncover Security Weakness: Stress testing helps to uncover the security
vulnerabilities that may enter into the system during the constant peak load and
compromise the system.
• To Ensures data integrity: Stress testing helps to determine the application’s data
integrity throughout the extreme load, which means that the data should be in a
dependable state even after a failure.

Stress Testing Process:

The stress testing process is divided into 5 steps:

1. Planning the stress test: This step involves gathering the system data, analyzing the
system, and defining the stress test goals.
2. Create Automation Scripts: This step involves creating the stress testing automation
scripts and generating the test data for the stress test scenarios.
3. Script Execution: This step involves running the stress test automation scripts and
storing the stress test results.
4. Result Analysis: This phase involves analyzing stress test results and identifying the
bottlenecks.
5. Tweaking and Optimization: This step involves fine-tuning the system and
optimizing the code with the goal meet the desired benchmarks.

Examples for stress testing : A web server may be stress tested using scripts, bots, and various
tools to observe the performance of a web site during peak loads. These attacks generally are
under an hour long, or until a limit in the amount of data that the web server can tolerate is found.

Types of Stress Testing:

1. Server-client Stress Testing: Server-client stress testing also known as distributed
stress testing is carried out across all clients from the server.
2. Product Stress Testing: Product stress testing concentrates on discovering defects
related to data locking and blocking, network issues, and performance congestion in a
software product.

Panimalar Engineering College Chennai City Campus 5

 3. Transactional Stress Testing: Transaction stress testing is performed on one or more
transactions between two or more applications. It is carried out for fine-tuning and
optimizing the system.
4. Systematic Stress Testing: Systematic stress testing is integrated testing that is used
to perform tests across multiple systems running on the same server. It is used to
discover defects where one application data blocks another application.
5. Analytical Stress Testing: Analytical or exploratory stress testing is performed to test
the system with abnormal parameters or conditions that are unlikely to happen in a real
scenario. It is carried out to find defects in unusual scenarios like a large number of
users logged at the same time or a database going offline when it is accessed from a
website.
6. Application Stress Testing: Application stress testing also known as product stress
testing is focused on identifying the performance bottleneck, and network issues in a
software product.

Stress Testing Tools:

1. Jmeter: Apache JMeter is a stress testing tool is an open-source, pure Java-based
software that is used to stress test websites.
2. LoadNinja: LoadNinja is a stress testing tool that enables users to develop codeless
load tests
3. WebLoad: WebLoad is a stress testing tool that combines performance, stability, and
integrity as a single process for the verification of mobile and web applications.
4. Neoload: Neoload is a powerful performance testing tool that simulates large numbers
of users and analyzes the server’s behavior.
5. SmartMeter: SmartMeter is a user-friendly tool that helps to create simple tests
without coding. It has a graphical user interface and has no necessary plugins.

Metrics of Stress Testing:

Metrics are used to evaluate the performance of the stress and it is usually carried out at the
end of the stress scripts or tests. Some of the metrics are given below.
1. Pages Per Second: Number of pages requested per sec and number of pages loaded per sec.
2. Pages Retrieved: Average time is taken to retrieve all information from a particular page.
3. Byte Retrieved: Average time is taken to retrieve the first byte of information from the page.
4. Transaction Response Time: Average time is taken to load or perform transactions
between the applications.
5. Transactions per Second: It takes count of the number of transactions loaded per
second successfully and it also counts the number of failures that occurred.
6. Failure of Connection: It takes count of the number of times that the client faced
connection failure in their system.
7. Failure of System Attempts: It takes count of the number of failed attempts
8. Rounds: It takes count of the number of test conditions executed by the clientssuccessfully
and it keeps track of the number of rounds failed.

Advantages of Stress Testing

 Determines the behavior of the system
 Ensure failure does not cause security issues
 Makes system function in every situation
 Improving Decision Making
 Increasing Stakeholder confidence

Panimalar Engineering College Chennai City Campus 6

Disadvantages of Stress Testing
 Manual stress testing is complicated
 Good scripting knowledge required.
 Need for external resources
 Additional tool required in case of open-source stress testing tool
4.4 Difference between Load Testing and Stress testing
Load Testing Stress testing
Load Testing is a type of performance Stress Testing is performed to test the
testing that determines the performance of a robustness of the system or software
different application under real-life-based application under extreme load.
load conditions.
In load testing load limit is the threshold of a In stress testing load limit is above the
break threshold of a break.
Huge number of users. There are too many users and too much data.

4.5 VOLUME TESTING

Volume Testing is a type of software testing which is carried out to test a software
application with a certain amount of data. The amount of data used in volume testing
could be a database size or it could also be the size of an interface file that is the subject of
volume testing. While testing the application with a specific database size, database is
extended to that size and after that the performance of the application is tested. When an
application needs interaction with an interface file this could be either reading or writing the
file or same from the file. A sample file of the size required is created and then functionality
of the application istested with that file in order to test the performance.
In volume testing a huge volume of data is acted upon the software. It is basically
performed to analyze the performance of the system by increasing the volume of data in the
database. Volume testing is performed to study the impact on response time and behavior of
the system when the volume of data is increased in the database. Volume Testing is also
known as Flood Testing.

Objectives (Goals) of Volume Testing

• To recognize the problems that may be created with large amount of data.
• To check the system’s performance by increasing the volume of data in the database.
• To find the point at which the stability of the system reduces.
• To identify the capacity of the system or application.
• To create the scaling plans

Volume Testing Attributes

• System’s Response Time: During the volume testing, the response time of the system
or the application is tested. It is also tested whether the system responses within the
finite time or not. If the response time is large then the system is redesigned.
• Data Loss: During the volume testing, it is also tested that there is no data loss. If there
is data loss some key information might be missing.
• Data Storage: During the volume testing, it is also tested that the data is stored
correctly or not. If the data is not stored correctly then it is restored accordingly in
proper place.
• Data Overwriting: In volume testing, it is tested that whether the data is overwritten
without giving prior information to the developer. If it so then developer is notified.

Panimalar Engineering College Chennai City Campus 7

 Various Difficulties that Volume testing face :

Volume testing has to handle big volumes of data . It is difficult to maintain the database that
has a strong structure. Also reviewing the data types and connections between them is difficult.

Example of volume testing:

A company might artificially grow their database to a certain size and test how the application
performs, a media company might test their digital asset management system by uploading
thousands of large video and image files

Advantages of Volume Testing

• Volume testing is helpful in saving maintenance cost
• Volume testing is also helpful in a rapid start for scalability plans.
• Volume testing also helps in early identification of bottlenecks.
• Volume testing ensures that the system is capable of real world usage.

Disadvantages of Volume Testing

• More number of skilled resources are needed to carry out this testing.
• It is sometimes difficult to prepare test cases
• It is a time consuming technique since it requires lot of time to decide the number of
volume of data and test scenarios.
• It is bit costly.
• It is not possible to have the exact break down of memory used in the real world
application.

Volume testing tools

HammerDB - It is an open-source tool used in the global database industry
DbFit - The DbFit tests can be used as existing executable documentation of system behavior.
JdbcSlim- used by Developers, Testers, and Business Users who know SQL language.
NoSQLMap- designed to automatically insert outbreaks to evaluate the threat.

4.6 FAIL-OVER TESTING

Software products/services are tested multiple times before delivery to ensure that it is
providing the required service. Testing before delivery doesn’t guarantee that no problem will
occur in the future. Even sometimes the software application fails due to some unwanted event
due to network issues or due to server-related problems. Failover testing aims to respond to
these types of failures. Suppose that the PC gets off due to some technical issue, and on
restarting we open the browser, then a pop-up is shown saying “Do you want to restore all
pages?” On clicking restore, all tabs are restored. The process of ensuring such restorations
after failure is known as FAILOVER TESTING.
What is Failover Testing :
Failover testing is a technique that validates if a system can allocate extra resources and
back up all the information and operations when a system fails abruptly due to some
reason. This test determines the ability of a system to handle critical failures and handle
extraservers.

Panimalar Engineering College Chennai City Campus 8

 It is preferred that testing should be performed by servers. Active-active and active-
passive standby are the two most common configurations. Both the techniques achieve failover
in a very different manner but both of them are performed to improve the server’s reliability.
For example, if we have three servers, one of them fails due to heavy load, and then two
situations occur. Either that failed server will restart on its own or in another situation when
the failed server cannot be restarted, the remaining servers will handle the load. Such
situations are tested during this test.

Factors to be considered before Performing Failover Testing:

1. The budget has to be the first thing to be taken into consideration before thinking about
performing the Failover test.
2. The budget is connected to the frameworks that might crash or break down under
pressure/load.
3. Always keep in mind how much time it will take to fix all the issues caused by the
failure of the system.
4. Note down the most likely failures and organize the outcomes according to how much
harm is caused by the failure.

Factors to be considered while Performing Failover Testing:

1. Keep a plan of measures to be taken after performing a test.
2. Focus on the execution of the test plan.
3. Set up a benchmark so that performance requirements can be achieved.
4. Prepare a report concerning issue requirements and/or requirements of the asset.

Working of Failover testing:

1. Consider the factors: Before performing failover testing consider the factors like
budget, time, team, technology, etc.
2. Analysis on failover reasons and design solutions: Determine probable failure
situations that the system might experience. Examine the causes of failure, including
software bugs, hardware malfunctions, network problems, etc. It provides fixes for any
flaws or vulnerabilities found in the failover procedure.
3. Testing failover scenarios: It develops extensive test cases to replicate various
failover scenarios. This covers both unplanned failovers (system or component
failures) and scheduled failovers (maintenance). Test cases ought to address many
facets of failover, such as load balancing, user impact, network rerouting, and data
synchronization.
4. Executing the test plan: To reduce the impact on production systems, carry out the
failover test plan in a controlled setting. Keep an eye on how the system behaves during
failover to make sure it satisfies the recovery point and recovery time objectives.

Panimalar Engineering College Chennai City Campus 9

 5. Detailed report on failover: Keep a record of the failover testing findings, including
any problems you ran across, how long it took to failover and how it affected
customers or services. Assess problems according to their severity and offer
suggestions for improvements.
6. Necessary actions based on the report: Distribute the report on the failover test to
all concerned parties, such as project managers, developers, and system
administrators. Determine what needs to be done and prioritize it based on the
report’s conclusions. This might involve fixing found flaws in the system, updating
failover setups or improving the documentation.

Benefits of Failover Testing:

1. Determines Vulnerabilities and Weaknesses.
2. Verifies Redundancy Procedures
3. Improving the User Experience
4. Encourages Compliance
5. Encourages Continuous Improvement

Challenges in Failover Testing

 Complex System Dependencies: Modern software is often a web of interconnected
services, making failover scenarios complex.
 Data Synchronization Issues: Ensuring data remains consistent across primary and
backup systems can be tricky.
 Resource Allocation: Backup systems need to be as robust as primary systems, which
can be resource-intensive.
 Network Configuration: Failover testing often involves intricate network setups that can
be difficult to manage.

● Example of Failover Testing: A bank needs to ensure its online banking system can
handle server failures without affecting customer transactions. The testing team
simulates a server crash in the primary data center while monitoring how quickly
transactions are redirected to a backup server in a different location. The failover
process takes 30 seconds, during which some transactions are lost. The team
implements improvements. After re-testing, the failover time is reduced to 5 seconds,
meeting the bank's requirements

4.7 RECOVERY TESTING

Recovery testing is a type of system testing which aims at testing whether a
system can recover from failures or not. The technique involves failing the system and
then verifyingthat the system recovery is performed properly.
To ensure that a system is fault-tolerant and can recover well from failures, recovery
testing is important to perform. A system is expected to recover from faults and resume its
work within a pre-specified time period. Recovery testing is essential for any mission-critical
system, for example, the defense systems, medical devices, etc. In such systems, there is a
strict protocol that is imposed on how and within what time period the system should recover
from failure and how the system should behave during the failure.
A system or software should be recovery tested for failures like:
• Power supply failure
• The external server is unreachable
• Wireless network signal loss
• Physical conditions, etc.

Panimalar Engineering College Chennai City Campus 10

 Steps to be performed before executing a Recovery Test
1. Recovery Analysis – It is important to analyze the system’s ability to allocate extra
resources like servers or additional CPUs. This would help to better understand the
recovery-related changes that can impact the working of the system. Also, each of the
possible failures, their possible impact, their severity, and how to perform them should
be studied.
2. Test Plan preparation – Designing the test cases keeping in mind the environment and
results obtained in recovery analysis.

3. Test environment preparation – Designing the test environment according to the

recovery analysis results.
4. Maintaining Back-up – Information related to the software, like various states of the
software and database should be backed up. Also, if the data is important, then the
backing up of the data at multiple locations is important.
5. Recovery personnel Allocation – For the recovery testing process, it is important to
allocate recovery personnel who are aware and educated enough for the recovery
testing being conducted.
6. Documentation – This step emphasizes on documenting all the steps performed before
and during the recovery testing so that the system can be analyzed for its performance
in case of a failure.

Examples of Recovery Testing

• When a system is receiving some data over a network for processing purposes, we can
stimulate software failure by unplugging the system power. After a while, we can plug
in the system again and test its ability to recover and continue receiving the data from
where it stopped.
• Another example could be when a browser is working on multiple sessions, we can
stimulate software failure by restarting the system. After restarting the system, we can
check if it recovers from the failure and reloads all the sessions it was previously
working on.
• While downloading a movie over a Wifi network, if we move to a place where there is
no network, then the downloading process will be interrupted. Now to check if the
process recovers from the interruption and continues working as before, we move back
to a place where there is a Wifi network. If the downloading resumes, then the software
has a good recovery rate.

Types of Recovery Testing

• Database Recovery Testing: Evaluate the system’s capacity to recover from corrupted
or malfunctioning databases. In order to test how well the system can restore the
database to a consistent and useful condition, it involves intentionally destroying or
damaging it.
• Load and Stress Recovery Testing: Determine how effectively the system bounces
back from variables that affect performance, including heavy loads or stressful
situations. It helps in determining if the system is capable of handling higher loads and
in the event that it cannot, how soon it will resume normal operation after the load is
dropped.
• Crash Recovery Testing: Determine how well the system bounces back from a
hardware or software failure. To make sure the system can resume regular operations
without losing data, it can involve unexpected shutdowns, abrupt power failures or a
sudden halt of services.

Panimalar Engineering College Chennai City Campus 11

 •Security Recovery Testing: Examine the system’s resilience to security lapses, illegal
access, and other security-related events by conducting security recovery testing. It
guarantees that the system can recover from security breaches and helps discover
loopholes in the security procedures, reducing the impact of any unauthorized access.
• Environment Recovery Testing: Examine the software’s ability to adjust to changes
in dependencies or configurations in the environment. It guarantees that in the event of
modifications to the underlying structure or environmental circumstances, the system
can recover and go on operating as anticipated.
Advantages of Recovery Testing
• Improves the quality of the system by eliminating the potential flaws in the system
so that the system works as expected.
• Recovery testing is also referred to as Disaster Recovery Testing. A lot of companies
have disaster recovery centers to make sure that if any of the systems is damaged or
fails due to some reason, then there is back up to recover from the failure.
• Risk elimination is possible as the potential flaws are detected and removed from the
system.
• Improved performance as faults are removed, and the system becomes more reliable
and performs better in case a failure occurs.

Disadvantages of Recovery testing

• Recovery testing is a time-consuming process as it involves multiple steps and
preparations before and during the process.
• The recovery personnel must be trained as the process of recovery testing takes place
under his supervision. So, the tester needs to be trained to ensure that recovery testing
is performed in the proper way. For performing recovery testing, he should have enough
data and back up files to perform recovery testing.
• The potential flaws or issues are unpredictable in a few cases. It is difficult to point
out the exact reason for the same, however, since the quality of the software must be
maintained, so random test cases are created and executed to ensure such potential
flaws are removed.

Recovery testing Vs Fail over testing

Recovery testing Fail over testing
Recovery testing validates a system's ability Failover testing verifies the system's
to recover from failures capacity to switch to a backup system in
the event of a failure
Recovery testing aims to ensure data Failover testing aims to maintain
integrity and system restoration with continuous operation by switching to a
minimal downtime backup system.

4.8 CONFIGURATION TESTING

Configuration Testing is the type of Software Testing that verifies the
performance of the system under development against various combinations of software
and hardware to find out the best configuration under which the system can work
without any flaws or issues while matching its functional requirements.
What is Configuration Testing?
Configuration Testing is the process of testing the system under each configuration of
the supported software and hardware. Here, the different configurations of hardware and
software mean the multiple operating system versions, various browsers, various supported
drivers, distinct memory sizes, different hard drive types, various types of CPU, etc.

Panimalar Engineering College Chennai City Campus 12

 The various configurations are
1. Operating System : Win XP, Win 7 32/64 bit, Win 8 32/64 bit, Win 10, etc.
2. Database Configuration: Oracle, DB2, MySQL, MSSQL Server, Sybase etc.
3. Browser Configuration: IE 8, IE 9, FF 16.0, Chrome, Microsoft Edge etc.

Objectives of Configuration Testing:

1. Adaptability to Different Configurations: Check that the program’s basic features
work consistently and dependably in all configurations. Testing the behavior of the
program with different setups and settings is part of this process.
2. Evaluation of Stability: Examine the software’s stability under various configurations.
Find and fix any configuration-specific problems that might be causing crashes,
unstable systems or strange behavior.
3. Testing the User Experience: Assess the value and consistency of the user experience
across various setups. Make that the graphical user interface (GUI) of the software
adjusts to various screen sizes, resolutions and display settings.
4. Security Throughout Configurations: To make sure that sensitive data is kept safe,
test the software’s security features in various setups. Determine and fix any
vulnerabilities that might be configuration-specific.
5. Compatibility of Networks: Examine the software’s behavior with various network
setups. Evaluate its compatibility with various network types, speeds and latency.
6. Data Compatibility: Check if the program can manage a range of data
configurations, such as those from diverse sources, databases and file formats. Verify
the consistency and integrity of the data across various setups.

Configuration Testing Process:

Types of Configuration Testing:

1. Software Configuration Testing: Software configuration testing is done over the
Application Under Test with various operating system versions and various browser
versions etc. It is a time-consuming testing as it takes long time to install and uninstall
the various software which are to be used for testing.
2. Hardware Configuration Testing: Hardware configuration testing is typically
performed in labs where physical machines are used with various hardware connected
to them.

Step to Design the Test Cases to Run on Each Configuration:

1. Select and set up the next test configuration from the list.
2. Start the software.
3. Load in the file configtest.doc.
4. Confirm that the file is displayed correctly.
5. Print the document.
6. Confirm that there are no errors and that the printed document matches the standard.
7. Log any discrepancies as a bug.

Panimalar Engineering College Chennai City Campus 13

 Execute the Tests on Each Configuration:
We need to run the test cases and carefully log. Then report our results to the team
and to the hardware manufacturers if necessary. It’s often difficult and time-consuming to
identify the specific source of configuration problems. A tester need to work closely with the
programmers and white-box testers to isolate the cause and decide if the bugs you find are due
to your software or to the hardware.
If the bug is specific to the hardware, consult the manufacturer’s website for
information on reporting problems to them. They may ask you to send copies of your test
software, your test cases, and supporting details to help them isolate the problem.
Rerun the Tests Until the Results Satisfy Your Team - Initially a few configurations might
be tried, then a full test pass, then smaller and smaller sets to confirm bug fixes. Eventually
you will get to a point where there are no known bugs or to where the bugs that still exist are
in uncommon or unlikely test configurations. At that point, you can call your configuration
testing complete.
Advantages:
1. Improved User Experience
2. Cost-Effective
3. System Stability

Disadvantages:
1. Increased Complexity
2. Resource Intensive
3. Time-Consuming

4.9 COMPATIBILITY TESTING

Compatibility test aims to check the developed software application functionality on
various software, hardware platforms, network and browser etc. Compatibility testing is
software testing which comes under the non-functional testing category, and it is performed
on an application to check its compatibility (running capability) on different
platform/environments. This testing is done only when the application becomes stable. This
compatibility testing is very important in product production and implementation point of view
as it is performed to avoid future issues regarding compatibility. Developers, testers, product
managers, and customers are all involved in compatibility testing

Types of Compatibility Testing:

1. Software:
• Testing the compatibility of an application with an Operating System like Linux, Mac,
Windows
• Testing compatibility on Database like Oracle SQL server, MongoDB server
• Testing compatibility on different devices like in mobile phones, computers.
Types based on Version Testing:
There are two types of compatibility testing based on version testing
 Forward compatibility testing: When the behavior and compatibility of a software or
hardware is checked with its newer version then it is called as forward compatibility
testing.
 Backward compatibility testing: When the behavior and compatibility of a software
or hardware is checked with its older version then it is called as backward compatibility
testing.
2. Hardware: Checking compatibility with a particular size of
• RAM, ROM, Hard Disk, Memory Cards, Processor,Graphics Card

Panimalar Engineering College Chennai City Campus 14

 3. Smartphones / Mobiles : Checking compatibility with different mobile platforms like android,
iOS etc.
4. Network: Checking network compatibility with different
• Bandwidth
• Operating speed
• Capacity
5. Browser: Examines the application compatibility with several browsers including FireFox,
Google Chrome, IE, etc
6. Devices: Checking if the software is compatible with various devices including Bluetooth, USB
port, printers, Scanners, etc

How to perform Compatibility testing? (Eg)

 Testing the application in the same environment but having different versions. For
example, to test compatibility of Facebook application in your android mobile, first
check for the compatibility with Android 9.0 and then with Android 10.0 for the same
version of Facebook App.
 Testing the application in a same version but having different environment. For
example, to test compatibility of Facebook application in your android mobile , first
check for the compatibility with a Facebook application of lower version with a Android
10.0(or your choice) and then with a Facebook application of higher version with a same
version of Android.
Who is Involved in Compatibility Testing?
Here are the key individuals responsible for performing compatibility tests in software testing:

 QA Engineers: Primarily responsible for designing, executing, and analyzing test cases across
different hardware, software, and platforms.
 Developers: Collaborate with QA to identify and fix compatibility issues.
 Product Managers: Define compatibility requirements and prioritize testing efforts.
 End-users: Can provide valuable feedback on real-world compatibility issues.

Why compatibility testing is important? (Advantages)

1. It ensures complete customer satisfaction.
2. It provides service across multiple platforms.
3. Reduced risk of errors
4. Cost-effective

Compatibility testing defects: (Disadvantages)

1. Variety of user interface.
2. Changes with respect to font size.
3. Alignment issues.
4. Issues related to overlapping of content.

Tools for Compatibility Testing :

Lambda Test - The tool ensures that your application is able to function efficiently on all
desktop as well as mobile browsers.
Cross Browser Testing- This compatibility testing tool that enables manual, visual, as well as
Selenium tests across a number of mobile and desktop
Browser Stack- With BrowserStack, you can test websites on different Android and iOS devices
across various browsers.

Panimalar Engineering College Chennai City Campus 15

Browser Shots -BrowserShots enable testing websites across any operating system and browser.
Browserling –It is one of the cost-effective compatibility testing tools which enable easy, live,
and interactive browser compatibility testing

4.10 USABILITY TESTING

Usability Testing is the testing done from an end user’s perspective to determine if the
system is easily usable. Usability testing is generally the practice of testing how easy a
design is to use on a group of representative users. Usability testing is also referred to as
User Experience.
A very common mistake in usability testing is conducting a study too late in the design
process. If you wait until right before your product is released, you won’t have the time or
money to fix any issues – and you’ll have wasted a lot of effort developing your product the
wrong way.
The following qualities are tested in Usability testing to ensure user friendliness
 Easy to understand
 Easy to access
 Look and feel
 Faster to access
 Effective navigation

Phases of Usability Testing [ usability Testing Process]

There are five phases in usability testing which are followed by the system when usability
testing is performed. These are given below:

1. Prepare your product or design to test: The first phase of usability testing is choosing
a product and then making it ready for usability testing.
2. Find your participants: Generally, the number of participants that you need is based
on several case studies. Mostly, five participants can find almost as many usability problems as
you’d find using many more test participants.
3. Write a test plan: The main purpose of the plan is to document what you are going to
do, how you are going to conduct the test, what metrics you are going to find, the number of
participants you aregoing to test, and what scenarios you will use.
4. Take on the role of the moderator: The moderator plays a vital role that involves
building a partnership with the participant. Most of the research findings are derived by
observing the participant’s actions and gathering verbal feedback to be an effective
moderator.
5. Present final report: This phase generally involves combining your results into an
overall score and presenting it meaningfully to your audience.

Why do we need Usability Testing?

When software is ready, it is important to make sure that the user experience with the product
should be seamless. It should be easy to navigate and all the functions should be working
properly, the competitor’s website will win the race. Therefore, usability testing is performed.

Panimalar Engineering College Chennai City Campus 16

 The objective of usability testing is to understand customers’ needs and requirements and also
how users interact with the product (software). With the test, all the features, functions, and
purposes of the software are checked.
The primary goals of usability testing are – discovering problems (hidden issues) and
opportunities, comparing benchmarks, and comparison against other websites. The
parameters tested during usability testing are efficiency, effectiveness, and satisfaction.
Factors Affecting Cost of Usability Testing:
The testing cost will depend on the following factors:
1. No. of participants for testing.
2. Number of Days needed for testing.
3. Which type of testing.
4. The size of the team used for testing.
A tester should also remember to budget for the usability testing, making usability testing into
a product . The other factors that are needed are as follows:
• Rental cost: If you are not considering the equipment, you will need to ensure the
budget cost for all other equipment, and also need to allot the location for the testing
purpose. For example the rental room like a conference room which is used to perform
all operations.
• Recruiting Costs: Consider how and where you have recruited your participants. You
will need to allow the staff to engage a recruiting team to schedule participants based
on requirements.
• Participants Compensation based on: You will need to compensate the participants
for their time and travel purposes that also important to finding the testing budget.

Techniques and Methods of Usability Testing:

There are various types of usability testing that when performed lead to efficient software. But
few of them which are the most widely used have been discussed here.
1. Guerilla Testing
It is a type of testing where testers wander to public places and ask random users about the
prototype. It is the best way to perform usability testing during the early phases of the
product development process. Users mostly spare 5–10 minutes and give instant feedback on
the product. Also, the cost is comparatively low as you don’t need to hire participants. It is
also known as corridor or hallway testing.
2. Usability Lab
Usability lab testing is conducted in a lab environment where moderators (who ask for
feedback on the product) hire participants and ask them to take a survey on the product. This
test is performed on a tablet/desktop. The participant count can be 8-10 which is a bit costlier
than guerrilla testing as you need to hire participants, arrange a place, and conduct testing.
3. Screen or Video Recording
Screen or video recording kind of testing is in which a screen is recorded as per the user’s
action (navigation and usage of the product). This testing describes how the user’s mind runs
while using a product. This kind of testing involves the participation of almost 10 users for 15
minutes. It helps in describing the issues users may face while interacting with the product.

Generally, there are two studies in usability testing –

1. Moderated – the Moderator guides the participant for the changes required in the
product (software)
2. Unmoderated – There’s no moderator (no human guidance), participants gets a set of
questions on which he/she has to work.

Panimalar Engineering College Chennai City Campus 17

An example of usability testing : Asking a group of users to complete a task on a new e-
commerce website, like "find a pair of blue jeans in size medium," while observing their actions
and asking follow-up questions to understand if they encounter any difficulties navigating the
site, finding the product, or completing the purchase, allowing the designers to identify areas for
improvement in user experience
Advantages of Usability Testing
 Meet the User’s Expectations
 Avoid design flaws
 Product becomes Efficient
Disadvantages of Usability Testing
 Expensive and time consuming
 Usability Test Outcomes are Arguable
 Selecting a Target Group Can be Tricky

4.11 TESTING THE DOCUMENTATION

The works of a software tester aren’t constrained to just testing the software. It is his
duty to cover all the parts that make up the entire software product. Hence assuring the
documentation is correct is also a tester’s job.

Type of Software Documentation:

Here’s a list of software components that can be classified as documentation:
• Packaging text and graphics - This includes the box, carton, wrapping, and so on. The
documentation might contain screen shots from the software, lists of features, system
requirements, and copyright information.
• Marketing material, ads, and other inserts - These are all the pieces of paper you usually
throw away, but they are important tools used to promote the sale of related software, add-on
content, service contracts, and so on. The information for them must be correct for a customer
to take them seriously.
• Warranty/registration- This is the card that the customer fills out and sends in to register
the software. It can also be part of the software, being displayed onscreen for the user to read,
acknowledge, and complete online.
• EULA- It stands for End User License Agreement. This is the legal document that the
customer agrees to that says, among other things, that he won’t copy the software nor sue
the manufacturer if he’s harmed by a bug.
• Labels and stickers- These may appear on the media, on the box, or on the printed material.
There may also be serial number stickers and labels that seal the EULA envelope. The Fig
below is an example of a disk label and all the information that needs to be checked.

Figure Sample documentation on the disk label for the software tester to check.

Panimalar Engineering College Chennai City Campus 18

 • Installation and setup instructions. Sometimes this information is printed directly on the
discs, but it also can be included on the CD content
• User’s manual. The usefulness and flexibility of online manuals has made printed manuals
much less common than they once were. Most software now comes with a small, concise
“getting started”–type manual with the detailed information moved to online format. The
online manuals can be distributed on the software’s media or on a website or on both.
• Online help- Online help often gets intertwined with the user’s manual, sometimes even
replacing it. Online help is indexed and searchable, making it much easier for users to find
the information they’re looking for.
• Tutorials, wizards, and CBT (Computer Based Training). These tools blend programming
code and written documentation. They’re often a mixture of both content and high-level,
macro-like programming and are often tied in with the online help system. A user can ask a
question and the software then guides him through the steps to complete the task.
• Samples, examples, and templates -An example of these would be a word processor with
forms or samples that a user can simply fill in to quickly create professional-looking results.
A compiler could have snippets of code that demonstrate how to use certain aspects of the
language.
• Test cases- Numerous test cases are used to test the software.They are recorded in a document
• Test plan – The strategy of testing is clearly mentioned in the Test plan. Test Data – The test
data are used to compare the test case findings and for writing test cases

The Importance of Documentation Testing (Advantages)

• It improves usability.
• It improves reliability.
• It lowers costs. The reason is that users who are confused or run into unexpected problems
will call the company for help, which is expensive. Good documentation can prevent these
calls by adequately explaining and leading users through difficult areas.

What to Look for When Reviewing Documentation:

Table 4.1 is a simple checklist to use as a basis for building your documentation test cases.

Table : A Documentation Testing Checklist

Panimalar Engineering College Chennai City Campus 19

 Drawbacks of the Test Documents
 Maintaining the documents is a tiresome work
 If the document is inadequate , the quality of the application will be less
 Sometimes it might be very expensive and more than it’s worth

4.12 SECURITY TESTING

Security testing is used to discover the weaknesses, risks, or threats in the
software application and also help us to stop the attackfrom the outsiders and make sure
the security of our software applications.
The primary objective of security testing is to find all the potential ambiguities and
vulnerabilities of the application so that the software does not stop working. If we perform
security testing, then it helps us to identify all the possible security threats and also help the
programmer to fix those errors.
It is a testing procedure, which is used to define that the data will be safe and also
continue the working process of the software.

Principle of Security testing:

Availability: In this, the data must be retained by an official person, and they also guarantee
that the data and statement services will be ready to use whenever we need it.
Integrity: In this, we will secure those data which have been changed by the unofficial person.
The primary objective of integrity is to permit the receiver to control the data that is given by
the system. The integrity systems regularly use some of the similar fundamental approaches
as confidentiality structures. And also verify that correct data is conveyed from one
application to another.
Authorization: It is the process of defining that a client is permitted to perform an action and
also receive the services. The example of authorization is Access control.

Confidentiality: It is a security process that protects the leak of the data from the outsider's
because it is the only way where we can make sure the security of our data.
Authentication: The authentication process comprises confirming the individuality of a
person, tracing the source of a product that is necessary to allow access to the private
information or the system

Panimalar Engineering College Chennai City Campus 20

 Non-repudiation: It is used as a reference to the digital security. It a way of assurance that the
sender of a message cannot disagree with having sent the message and that the recipient
cannot repudiate having received the message.
The non-repudiation is used to ensure that a conveyed message has been sent and
received by the person who claims to have sent and received the message.

Key Areas in Security Testing:

While performing the security testing on the web application, we need to concentrate on the
following areas to test the application:

System software security: In this, we will evaluate the vulnerabilities of the application based
on different software such as Operating system, Database system, etc.
Network security: In this, we will check the weakness of the network structure, such as
policies and resources.
Server-side application security: We will do the server-side application security to ensure
that the server encryption and its tools are sufficient to protect the software from any
disturbance.
Client-side application security: In this, we will make sure that any intruders cannot operate
on any browser or any tool which is used by customers.

Types of Security testing:

Security Scanning: Security scanning can be done for both automation testing and manual
testing. This scanning will be used to find the vulnerability or unwanted file modification in a
web-based application, websites, network, or the file system. After that, it will deliver the
results which help us to decrease those threats. Security scanning is needed for those systems,
which depends on the structure they use.

Risk Assessment: To moderate the risk of an application, we will go for risk assessment. In
this, we will explore the security risk, which can be detected in the association. The risk can
be further divided into three parts, and those are high, medium, and low. The primary purpose
of the risk assessment process is to assess the vulnerabilities and control the significant threat.

Panimalar Engineering College Chennai City Campus 21

 Vulnerability Scanning: It is an application that is used to determine and generates a list of
all the systems which contain the desktops, servers, laptops, virtual machines, printers,
switches, and firewalls related to a network. The vulnerability scanning can be performed over
the automated application and also identifies those software and systems which have
acknowledged the security vulnerabilities.

Penetration testing: Penetration testing is a security implementation where a cyber-security

professional tries to identify and exploit the weakness in the computer system. The primary
objective of this testing is to simulate outbreaks and also finds the loophole in the system and
similarly save from the intruders who can take the benefits.
Security Auditing: Security auditing is a structured method for evaluating the security
measures of the organization. In this, we will do the inside review of the application and the
control system for the security faults.
Ethical hacking: Ethical hacking is used to discover the weakness in the system and also helps
the organization to fix those security loopholes before the hacker exposes them. The ethical
hacking will help us to increase the security position of the association because sometimes
the ethical hackers use the same tricks, tools, and techniques that nasty hackers will use, but
with the approval of the official person.
The objective of ethical hacking is to enhance security and to protect the systems from
malicious users' attacks.
Posture Assessment: It is a combination of ethical hacking, risk assessments, and security
scanning, which helps us to display the complete security posture of an organization.

Security testing tools:

We have various security testing tools available in the market, which are as follows:
SonarQube, ZAP, Netsparker, Arachni, IronWASP

4.13 TESTING IN THE AGILE ENVIRONMENT

o Agile Testing is a type of software testing that follows the principles of agile software
development to test the software application.
o All members of the project team along with the special experts and testers are involved
in agile testing.
o Agile testing is not a separate phase and it is carried out with all the development phases
i.e. requirements, design and coding, and test case generation. Agile testing takes place
simultaneously throughout the Development Life Cycle.
o Agile testers participate in the entire development life cycle along with development
team members and the testers help in building the software according to the customer
requirements and with better design and thus code becomes possible.

Panimalar Engineering College Chennai City Campus 22

 o Agile Testing has shorter time frames called iterations or loops. This methodology is
also called the delivery-driven approach because it provides a better prediction on the
workable products in less duration time.

Agile Testing Principles ( Features / Advantages )

 Quick feedback : In Agile Testing, the testing team gets to know the product
development and its quality for each and every iteration. Thus continuous feedback
minimizesthe feedback response time and the fixing cost is also reduced.
 Continuous Testing: Agile testing is not a different phase. It is performed alongside
the development phase. It ensures that the features implemented during that iteration are
actually done. Testing is not kept pending for a later phase.
 Involvement of all members: Agile testing involves each and every member of the
development team and the testing team. It includes various developers and experts.
 Less Documentation: Agile testers use reusable checklists to suggest tests and focus on
the essence of the test rather than the incidental details.
 Clean code: The defects that are detected are fixed within the same iteration. This
ensures clean code at any stage of development.
 Constant response: Agile testing helps to deliver responses or feedback on an ongoing
basis. Thus, the product can meet the business needs.
 Customer satisfaction: As the customers are exposed to the product throughout the
development process in agile testing, the customer can modify the requirements, and update
the requirements and the tests can also be changed as per the changed requirements.
 Test-driven: In agile testing, the testing needs to be conducted alongside the
development process to shorten the development time. But testing is implemented after the
implementationor when the software is developed in the traditional process.

Agile Testing Life Cycle

The agile testing life cycle has 5 different phases:
1. Impact Assessment: This is the first phase of the agile testing life cycle also known as
the feedback phase where the inputs and responses are collected from the users and
stakeholders. This phase supports the test engineers to set the objective for the next phase in
the cycle.
2. Agile Testing Planning: In this phase, the developers, customers, test engineers, and
stakeholders team up to plan the testing process schedules, regular meetings, and deliverables.
3. Release Readiness: This is the third phase in the agile testing lifecycle where the test
engineers review the features which have been created entirely and test if the features are ready
to go live or not and the features that need to be sent again to the previous development phase.
4. Daily Scrums: This phase involves the daily morning meetings to check on testing and
determine the objectives for the day. The goals are set daily to enable test engineers to
understand the status of testing.
5. Test Agility Review: This is the last phase of the agile testing lifecycle that includes
weekly meetings with the stakeholders to evaluate and assess the progress against the goals.

Panimalar Engineering College Chennai City Campus 23

 Agile Testing Strategies:
Agile testing has four strategies or stages that helps to enhance the quality of the product

1. Iteration 0
It is the first stage of the testing process and the initial setup is performed in this stage. The
testing environment is set in this iteration.
• This stage involves executing the preliminary setup tasks such as finding people for
testing, preparing the usability testing lab, preparing resources, etc.
• The business case for the project, boundary situations, and project scope are verified.
• Important requirements and use cases are summarized.
• Initial project and cost valuation are planned.
• Risks are identified.
2. Construction Iteration
It is the major phase of the testing and most of the work is performed in this phase. It is a set
of iterations to build an increment of the solution. This process is divided into two types:
a. Confirmatory testing: This type of testing concentrates on verifying that the system
meets the stakeholder’s requirements as described to the team to date and is performed by
the team. It is further divided into 2 types of testing:
Agile acceptance testing: It is the combination of acceptance testing and functional
testing. It can be executed by the development team and the stakeholders.
Developer testing: It is the combination of unit testing and integration testing and
verifies both the application code and database schema.
b. Investigative testing: Investigative testing detects the problems that are skipped or
ignored during confirmatory testing. In this type of testing, the tester determines the
potential problems in the form of defect stories. It focuses on issues like integration
testing, load testing, securitytesting, and stress testing.

3. Release End Game

This phase is also known as the transition phase. This phase includes the full system testing
and the acceptance testing. To finish the testing stage, the product is tested more relentlessly
while it is in construction iterations. In this phase, testers work on the defect stories. This phase
involves activities like:
• Training end-users.
• Support people and operational people.
• Marketing of the product release.
• Back-up and restoration.
• Finalization of the system and user documentation.

4. Production
It is the last phase of agile testing. The product is finalized in this stage after the removal of all
defects and issues raised.

Panimalar Engineering College Chennai City Campus 24

 Agile Testing Quadrants
The whole agile testing process is divided into four quadrants:

1. Quadrant 1 (Automated)
The first agile quadrat focuses on the internal quality of code which contains the test cases and
test components that are executed by the test engineers. All test cases are technology-driven
and used for automation testing. All through the agile first quadrant of testing, the following
testing can be executed:
• Unit testing / Component testing.

2. Quadrant 2 (Manual and Automated)

The second agile quadrant focuses on the customer requirements that are provided to the testing
team before and throughout the testing process. The test cases in this quadrant are business-
driven and are used for manual and automated functional testing. The following testing will be
executed in this quadrant:
• Pair testing.
• Testing scenarios and workflow.
• Testing user stories and experiences like prototypes.

3. Quadrant 3 (Manual)
The third agile quadrant provides feedback to the first and the second quadrant. This quadrant
involves executing many iterations of testing, these reviews and responses are then used to
strengthen the code. The test cases in this quadrant are developed to implement automation
testing. The testing that can be carried out in this quadrant are:
• Usability testing.
• Collaborative testing.
• User acceptance testing.
• Pair testing with customers.

4. Quadrant 4 (Tools)
The fourth agile quadrant focuses on the non-functional requirements of the product like
performance, security, stability, etc. Various types of testing are performed in this quadrant to
deliver non-functional qualities and the expected value. The testing activities that can be
performed in this quadrant are:
• Non-functional testing such as stress testing, load testing, performance testing, etc.
• Security testing.
• Scalability testing.
• Infrastructure testing.
• Data migration testing.

Panimalar Engineering College Chennai City Campus 25

 Challenges During Agile Testing (Disadvantages):
Changing requirements: Sometimes during product development changes in the
requirements or the specifications occur but when they occur near the end of the sprint, the
changes are moved to the next sprint and thus become the overhead for developers and testers.
Inadequate test coverage: In agile testing, testers sometimes miss critical test cases because
of the continuously changing requirements and continuous integration. This problem can be
solved by keeping track of test coverage by analyzing the agile test metrics.
Tester’s availability: Sometimes the testers don’t have adequate skills to perform API and
Integration testing, which results in missing important test cases.
Less Documentation: In agile testing, there is less or no documentation which makes the
task of the QA team more tedious.
Performance Bottlenecks: Sometimes developer builds products without understanding the
end-user requirements and following only the specification requirements, resulting in
performance issues in the product.
Skipping essential tests: In agile testing, sometimes agile testers due to time constraints and
the complexity of the test cases, put some of the non-functional tests on hold. This may
causesome bugs later that may be difficult to fix.

4.14 TESTING WEB AND MOBILE APPLICATIONS

Web Applications
Web applications refer to computer programs that run in a web browser. Commonly built with
the help of HTML5, CSS and JavaScript, web applications offer more interactivity than
websites and can be accessed via a desktop or laptop. The classic examples of web applications
include webmail, online stores and web banking.

Mobile Applications
A mobile application is a program that was built to be used on mobile devices (smartphones,
tablets and various wearables). Mobile apps are not as straightforward as desktop web apps
and fall into three varieties: mobile web, native and hybrid apps.
Mobile web applications
A mobile web application is a program that can be accessed via a mobile browser, meaning
that you don’t have to download them to your device to start using them. Like web apps, mobile
web applications are usually built using JavaScript, CSS and HTML5; however, there is no
standard software kit. Contrary to other mobile applications, web apps for mobile use are easier
to build and test, but they’re usually much more primitive in terms of functionality.
Native applications
Native mobile applications run on the device itself, so you have to download them before
using them. Since they are platform- specific, native mobile apps are built using specific
languages and integrated development environments (IDEs). For example, Android native
applications are developed using Java and Android Studio or Eclipse IDE. At the same time,
to build an app for an Apple device, you’ll need to use Objective-C or Swift and the XCode
IDE. Native apps are secure, integrate with the hardware perfectly and have the best UI/UX
experience.
Hybrid applications
Hybrid apps combine the characteristics of native and mobile web apps. Built with the help of
the “standard web” stack (JavaScript, CSS and HTML5), they are then wrapped in a native
environment, so you can use the same code for different platforms. While running on your
mobile browser, hybrid applications are downloadable and have access to your camera, GPS,
contact list, etc. Though such applications are easier to build and maintain, they are slower and
offer less advanced functionality than their native counterparts.

Panimalar Engineering College Chennai City Campus 26

 Types of Mobile App Testing and Web App Testing
Whether it comes to testing web or mobile applications, the aim is to ensure that an app is user-
friendly and functions properly under different circumstances. Furthermore, both application
testing varieties include common types of testing listed below:
• Functional testing
• User-interface testing
• Usability testing
• Configuration and Compatibility testing
• Security testing
• Performance testing
• Database testing

1. Functional Testing
Functional testing involves checking of the specified functionality of a web application.
Functional test cases for web applications may be generated using boundary value analysis,
equivalence class testing, decision table testing and many other techniques.
Example: Let us consider the eCommerce application sells products such as computers,
mobile phones, cameras, , etc. The home page of this web application is given in Fig below:

Figure : Homepage of online shopping web application

Table below presents some sample functional test cases of the order process form of an
online shopping website.

Table: Sample functional test cases of order process of an online shopping webapplication

2. User-interface Testing
User interface testing tests that the user interaction features work correctly. These
features include hyperlinks, tables, forms, frames and user interface items such as text fields,
radio buttons, check boxes, list boxes, combo boxes, command buttons and dialog boxes.

Panimalar Engineering College Chennai City Campus 27

 User interface testing ensures that the application handles mouse and keyboard events
correctly and displays hyperlinks, tables, frames, buttons, menus, dialog boxes, error message
boxes, and toolbars properly.
2.1. Navigation Testing
Navigation testing investigates the proper functioning of all the internal and external links.
Navigation testing must ensure that websites provide consistent, well-organized links and
should also provide alternative navigation schemes such as search options and site maps. The
placement of navigation links on each page must be checked. Search based navigation facility
must also be thoroughly tested and search items should be consistent across one page to
another. All the combinations of keywords and search criteria must be verified in navigation
testing. Table below presents test cases for navigation testing for an online shopping website

Table :Navigation testing test cases for online shopping website

Manual checking of hyperlinks can be very time consuming. There are various online
tools available for checking broken links, accuracy and availability of links and obtaining
advice on search engines. Some tools for navigation testing include Link checker, Dead Links,
LinkTiger.

2.2 Form Based Testing

Websites that include forms need to ensure that all the fields in the form are working properly.
Form-based testing involves the following issues:
1. Proper navigation from one field of the form to a
2. nother using the tab key.
3. Ensures that the data entered in the form is in a valid format.
4. Checks that all the mandatory fields are entered in the form.

3. Usability Testing
Usability testing refers to the procedure employed to evaluate the degree to which the software
satisfies the specified usability criteria.

Panimalar Engineering College Chennai City Campus 28

 4. Configuration and Compatibility Testing
One of the significant challenges of web testing is that it must ensure the proper
functioning of a web application on all the supported platforms and suitable environments.
Configuration testing determines the behaviour of the software with respect to various
configurations whereas compatibility testing determines whether the web application behaves
as expected with respect to various supported configurations.
5. Security Testing
Security is the procedure used to protect information from various threats. It is very
important to protect sensitive and critical information and data while communicating over the
network. The user wants implementation of a safeguard to protect personal, sensitive and
financial information. We want data to be accurate, reliable and protected against unauthorized
access.
Security involves various threats such as unauthorized users, malicious users, message
sent to an unintended user, etc. The primary requirement of security includes:
i. Authentication: Is the information sent from an authenticated user?
ii. Access Control: Is data protected from unauthorized users?
iii. Integrity: Does the user receive exactly what is sent?
iv. Delivery: Is the information delivered to the intended user?
v. Reliability: What is the frequency of a failure? How much time does the network take to
recover from a failure? What measures are taken to counter catastrophic failure?
6. Performance Testing The goal of performance testing is to evaluate the application’s
performance with respect to real world scenarios. The following issues must be addressed
during performance testing:
i. Performance of the system during peak hours (response time, reliability and
availability).
ii. Points at which the system performance degrades or system fails.
iii. Impact of the degraded performance on the customer loyalty, sales and profits.
6.1 Load Testing
Load testing involves testing the web application under real world scenarios by
simulating numerous users accessing the web application simultaneously. It tests the web
application by providing it maximum load.
6.2 Stress Testing
Stress testing involves execution of a web application with more than maximum and
varying loads for long periods.
7. Database Testing
In web applications, many applications are database driven, for example, e-commerce
related websites or business-to-business applications. It is important for these applications to
work properly and provide security to the user’s sensitive data such as personal details and
credit card information. Testing data-centric web applications is important to ensure their error-
free operation and increased customer satisfaction.
For example, consider the example for purchasing items from an online store. If the
user performs a search based on some keywords and price preferences, a database query is
created by the database server. Suppose due to some programming fault in the query, the query
does not consider the price preferences given by the customer, this will produce erroneous
results. These kinds of faults must be tested and removed during database testing.

Table : shows sample test cases based on a user operation in an online shopping website.

Panimalar Engineering College Chennai City Campus 29

Difference between Mobile App and Web App
S.No Mobile App Web App
1 These are software programs that are These are software programs that are used on
used on mobile devices. computer.
2 New applications can be downloaded
Applications will be updated on website.
from app store.
3 It is not easy to create responsive
It is easy to code relative design for large
design for small screen devices such
screen devices such as desktop and laptop.
as mobile devices, tablets.
4 Web applications are developed for shorter
Mobile applications are developed for
range of users as compared to
broader range of users.
mobile applications
5 Mobile storage capacity is less than Desktop or laptop has larger storage capacity
desktop or laptop as compare to mobile.

Tools for Mobile App and Web App Testing

The reasons for choosing the right tool for test automation are a higher level of test coverage,
better reliability and faster test execution

Tools for Web App Testing

 Selenium
Selenium is a powerful open-source automated testing framework consists of Selenium IDE,
Selenium WebDriver and Selenium Grid. Selenium supports multiple programming languages
for script creation, allows users to record and re-run saved scripts and works well on different
browsers and operating systems.

Tools for Mobile App Testing

 Appium
Appium is a black-box mobile app testing tool. Based on Selenium, it is an open-source tool
for testing hybrid, web and native Android and iOS mobile applications..
 Espresso
Espresso is a UI quality assurance framework designed by Google for white box testing. Since
it was created to test Android native applications, Espresso tests can be written in Java and
Kotlin, the programming languages used to develop Android applications.

Selenium Appium Espresso

Platform Desktop Both Android & Android

Type browsers iOS
App type Web Native, web and Native, web
Hybrid and hybrid
Areas Functional, Functional, UI
to test regression regression, UI
Scripting Java, C#, Perl, Python, Java, C#, Python, PHP, Java
language JavaScript, Ruby, JavaScript
Ruby, PHP

License Open-source Open-source Open-source

Type

Panimalar Engineering College Chennai City Campus 30