Software Testing

What is Software Testing?

As per ANSI/IEEE 1059, Testing in Software Engineering is a process of evaluating a software
product to find whether the current software product meets the required conditions or not. The testing
process involves evaluating the features of the software product for requirements in terms of any missing
requirements, bugs or errors, security, reliability and performance.
Two different components constitute together as Software Testing. They are Software Validation and
Software Verification.

What are the benefits of Software Testing?

Here are the benefits of using software testing:
 Product quality: It is an essential requirement of any software product. Testing ensures a quality
product is delivered to customers.
 Customer Satisfaction: The main aim of any product is to give satisfaction to their customers.
UI/UX Testing ensures the best user experience.
 Moreover, early software testing uncovers problems before a product goes to market. The sooner
development teams receive test feedback, the sooner they can address issues such as:
✓ Architectural flaws
✓ Poor design decisions
✓ Invalid or incorrect functionality
✓ Security vulnerabilities
✓ Scalability issues

Verification and Validation

Validation and verification are two essential concepts in the software development lifecycle that
ensure a product meets its requirements and functions correctly. Here’s how they differ, along with
examples in the context of a Library Management System:

Verification: Verification is the process of evaluating work products (like documents, design, and
code) to determine if they meet the specified requirements at a certain stage of development. It
answers the question, "Are we building the product right?"
Example in Library Management:
 Code Reviews: Reviewing the code for the Library Management System to ensure it follows to
coding standards and meets design specifications.

Validation: Validation is the process of evaluating the final product to determine if it meets the user’s
needs and requirements. It answers the question, "Are we building the right product?"
Example in Library Management:
 User Acceptance Testing (UAT): The system allows users to easily search for books, check
availability, and borrow items seamlessly.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 1
 Software Testing
Summary of Differences
 Verification: Focuses on internal processes and development adherence to specifications. It is often
done through reviews and inspections before the product is completed.
 Validation: Focuses on the end product’s usability and whether it meets user needs and
requirements. It typically occurs after the product is developed.

Software Inspections
Software inspections are a formal process of examining software artifacts (such as requirements
documents, design specifications, or source code) to identify defects, ensure adherence to standards,
and verify that the product meets its specifications. Inspections are typically conducted by a group of
peers and involve a structured approach.
Examples in the Context of a Library Management System:
1. Requirements Inspection:
o Purpose: To ensure that the requirements for the Library Management System are clear,
complete, and feasible.
o Example: A team reviews the requirements document to check if all functionalities (like
searching for books, managing loans, and handling returns) are accurately described and
that there are no ambiguous terms or missing requirements.
2. Design Inspection:
o Purpose: To verify that the system architecture and design meet the specified requirements
and that the design is robust.
o Example: Inspecting the system architecture diagram to ensure that components (like the
user interface, database, and backend services) interact correctly. The team may look for
design flaws, such as a lack of scalability or issues in data flow.

Relation between the software Inspection and testing

In summary, while software inspections focus on improving the quality of the design and code before
execution, testing focuses on validating that the final product works as intended. Together, they create
a robust framework for ensuring software quality.

Figure: Relation between the software Inspection and testing

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 2
 Software Testing

Job Description: Software Tester

The software tester will conduct tests, analyze the results, and report observations to the design team.
You may also interact with clients in order to understand the requirements of the product.
To be successful as a software tester, you should have a working knowledge of software and test design,
the capability to run through tests, and the ability to analyze the results. Ultimately, the software tester
should be result-driven, have good communication skills, and up-to-date knowledge of software
programming and software test design.
Key Responsibilities:
 Reviewing software requirements and preparing test scenarios.
 Preparing Test Scenarios, test cases
 Executing tests on software usability.
 Analyzing test results on database impacts, errors or bugs, and usability.
 Preparing reports on all aspects related to the software testing carried out and reporting to the
design team.
 Interacting with clients to understand product requirements.
 Participating in design reviews and providing input on requirements, product design, and
potential problems.
 Conducting Review Meetings within the Team

Types of Software Testing: (Manual-Automation Aspect)

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 3
 Software Testing

Manual and Automated Testing

Below are a few of the significant differences between manual and automation testing:

Facts Manual Testing Automated Testing

Definition Manual testing is carried out by a Automation testing involves the use of
tester manually without the use of any special automation tools and requires a
kind of automation tool. huge investment of money and
resources.

Execution Testers manually execute test cases. Uses tools to schedule and execute the
test cases.

Time and cost The manual test takes up lots of time Automated testing: since the test cases
and implies a high cost. are automated, it saves time and is very
low.

Nature The process is such that it is Since the automation tool handles the
repetitive and boring. execution, the tester skips the boring
part.

Reliability Low reliability as manual High accuracy since all test cases are
and verification is prone to human error automated and executed by tools
Accuracy

User interface More user-friendly and guarantees Does not guarantee user-friendliness
improved customer experience or good customer experience.

Types of Manual Testing

1. White-box Testing: White-box testing involves testing the internal structures or workings of an
application. The tester has knowledge of the code, algorithms, and architecture.

Example: A developer tests a new sorting algorithm implemented in a software application by

writing unit tests that specifically check the function’s logic, such as verifying if the algorithm
correctly sorts a list of integers, handles edge cases, and ensures performance under different input
conditions.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 4
 Software Testing
2. Black-box Testing: Black-box testing focuses on testing the functionality of an application without
peering into its internal structures or workings. The tester does not need knowledge of the code and
tests the software against its specifications.

Example: A QA tester tests a web application’s login feature by entering valid and invalid
credentials and checking if the system responds correctly (e.g., allowing access for valid credentials
and showing an error message for invalid ones) without knowing how the login mechanism is
implemented behind the scenes.
3. Grey-box Testing: Grey-box testing is a combination of white-box and black-box testing. The tester
has partial knowledge of the internal workings of the application, which helps in designing test cases.
Example: A tester has access to the application’s database schema and some code snippets. They
might check whether the application correctly retrieves data from the database after performing a
search, using their understanding of both the front-end functionality and back-end processes.

Different techniques of White-box testing

White-box testing involves examining the internal structures or workings of an application. Here are
several common techniques used in white-box testing, along with examples for each:

1. Unit Testing: Unit testing involves testing individual components or functions of a software
application in isolation to ensure they work correctly.
Example: In a library management system, a developer tests the function that calculates overdue fines.
They create tests to verify that the function correctly calculates fines for different overdue durations,
such as 1 day, 5 days, and 30 days.

2. Control Flow Testing: Control flow testing examines the control flow of the program to ensure all
paths are tested.
Example: In a library management system's book borrowing function, the developer creates tests to
cover all possible paths through the code:
 User borrows a book successfully.
 User tries to borrow a book that is already borrowed.
 User tries to borrow more than the allowed limit.

3. Data Flow Testing: This technique focuses on the lifecycle of data within the program, ensuring
that variables are correctly defined, used, and released.
Example: A developer analyzes the data flow in the function responsible for searching books. They
check if all variables are initialized before use and if any are unnecessarily retained after they are no
longer needed.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 5
 Software Testing
4. Path Testing: Path testing involves testing all possible paths in a program to ensure complete
coverage of the code.
Example: In the library management system, the developer identifies multiple paths in a function that
checks user credentials:
 Path for valid username and password.
 Path for invalid username.
 Path for invalid password. They create tests to cover each of these paths.

Different techniques of Black-box testing

a) Equivalence Partitioning: Testers can divide possible inputs into groups or “partitions”, and test
only one example input from each group.
Example of equivalence partitioning:
The discount is calculated depending on the total amount of the shopping cart. If the total amount is in
the range of $100–$200, the discount is 10%. If the total amount is in the range of $201–$500, the
discount is 20%. If the total amount is more than $500, the discount is 30%. In this scenario, we can
identify three valid partitions and one invalid partition for the amount under $100.

Example of equivalence partitioning

To apply equivalence partitioning, we can take one value from each partition: $140 from the first valid
partition, $370 from the second valid partition, $530 from the third valid partition, and $78 from the
invalid partition. Now we have four test cases and have achieved 100% coverage because all defined
partitions are covered.

b) Boundary Value Analysis: This technique focuses on testing the boundaries between partitions, as
errors often occur at the edges.

Example of two-value boundary analysis:

Let’s look at the same discount scenario from earlier. The discount is calculated based on the total
amount of the shopping cart. If the total amount is in the range of $100–$200, the discount is 10%. If
the total amount is in the range of $201–$500, the discount is 20%. If the total amount is more than
$500, the discount is 30%. With a simple illustration, we can define the boundaries very easily.

Example of two-value boundary analysis

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 6
 Software Testing
To apply two-value boundary analysis we will test the minimum and maximum value of each boundary:
$99, $100, $200, $201, $500, and $501. We have six test cases and we will achieve 100% coverage by
testing them because all defined boundaries are covered.
c) Decision Table Testing
The decision table testing technique is used to visually present the different combinations of inputs and
outputs and is more focused on business rules. We use decision table testing in situations where different
combinations of test input conditions result in different outputs. With this technique, we identify the
conditions and the resulting actions of the testing object and present them in a table.
Example of decision table testing:
The discount is calculated depending on the total amount of the shopping cart and whether the customer
is a member. If the total amount is more than $200 and the customer is a member, the discount is 10%.
If the customer is not a member, there is no discount regardless of the total amount in the shopping cart.
To use the decision table testing technique, we first need to identify the conditions. In this case, the
conditions are the total amount in the shopping cart and whether a customer is a member or not. Next,
we need to identify all combinations of these conditions. We will mark them with Y and N.

The next step is to identify the correct actions/outcomes for each combination. We will use Y and N to
mark them.

As the final step, we write test cases for each rule in the table. In this table we have four rules which
means we will have four test cases:

Test case 1 for Rule 1: The total amount is more than $200 and the customer is a member, the
outcome will be a 10% discount off the total amount in the shopping cart.
Test case 2 for Rule 2: The total amount is more than $200, but the customer is not a member. There
will be no discount.
Test case 3 for Rule 3: The customer is a member, but the total amount is not more than $200. There
will be no discount.
Test case 4 for Rule 4: The total amount is not more than $200 and the customer is not a member.
There will be no discount.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 7
 Software Testing

Functional Testing:
Functional testing is performed to verify that all the features developed are according to the
functional specifications, and it is performed by executing the functional test cases written by the QA
team, in functional testing phase, system is tested by providing input, verifying the output and
comparing the actual results with the expected results.

There are different Levels of Functional Testing out of which the most important are
1. Unit Testing: Unit testing involves testing individual components or functions of a software
application in isolation to ensure they work correctly.
Example: In a library management system, a developer tests the function that calculates overdue
fines. They create tests to verify that the function correctly calculates fines for different overdue
durations, such as 1 day, 5 days, and 30 days.
2. Integration Testing: Integration testing checks the interactions between integrated units or
components to ensure they work together as expected.
Example: In the same library management system, integration testing may involve checking the
interaction between the user authentication module and the book borrowing module to verify that
users can borrow books only after logging in successfully.
3. System Testing: System testing evaluates the complete and integrated software system to verify
that it meets specified requirements.
Example: Testing the entire library management system as a whole to ensure all functionalities
(e.g., searching for books, borrowing, and returning) work correctly together and meet user
requirements.
4. Acceptance Testing: Acceptance testing assesses whether the software meets business needs and
is ready for delivery. This is often performed by end-users.
Example: In the library management system, users may conduct acceptance testing by using the
system to perform typical tasks (like searching for books and managing accounts) and ensuring it
meets their expectations before final approval.

a. Alpha Testing: Alpha testing is an internal testing phase conducted by developers or internal
teams to identify bugs before the software is released to external testers.
Example: Before releasing the library management system to a select group of users, the
development team conducts alpha testing to identify and fix any major issues in a controlled
environment.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 8
 Software Testing
b. Beta Testing: Beta testing involves releasing the software to a limited audience outside the
development team to gather feedback and identify any remaining issues.
Example: The library management system is released to a group of librarians and users to test
in a real-world environment, allowing them to provide feedback on functionality and usability
before the final release.

Non- Functional Testing:

Non-Functional testing is a testing technique which does not focus on functional aspects and mainly
concentrates on the nonfunctional attributes of the system such as memory leaks, performance or
robustness of the system. Non-Functional testing is performed at all test levels. There are many Non-
Functional Testing Techniques out of which the most important are:
i. Load Testing: Load testing evaluates the system's performance under expected user loads to
ensure it can handle the required number of concurrent users and transactions.
Example: In a library management system, load testing could involve simulating 500 users
simultaneously searching for books, borrowing items, and returning books to check if the system
maintains acceptable response times and performance without crashing.

ii. Stress Testing: Stress testing determines the system's behavior under extreme conditions,
beyond normal operational capacity, to identify its breaking point and how it recovers.
Example: For the library management system, stress testing might involve increasing the number
of simultaneous users to 1,000 or more and observing how the system handles the load, including
measuring how it performs when it exceeds its maximum capacity, such as during a high-demand
period like the start of a school semester.

iii. Security Testing: Security testing identifies vulnerabilities in the system to ensure data
protection and that the application is secure against threats and attacks.
Example: In a library management system, security testing could involve testing for common
vulnerabilities such as SQL injection, cross-site scripting (XSS), and verifying that sensitive user
information (like account details and borrowing history) is securely encrypted and protected
from unauthorized access.

iv. Recovery Testing: Recovery testing assesses how well a system can recover from crashes,
hardware failures, or other disruptions, ensuring that data integrity is maintained.
Example: For the library management system, recovery testing might involve simulating a power
failure or server crash while a user is borrowing a book and then checking if the system can
restore the previous session accurately and that no data is lost after the restart.

v. Compatibility Testing: Compatibility testing ensures that the software functions correctly
across different devices, browsers, operating systems, and environments.
Example: For the library management system, compatibility testing would involve checking if
the application works properly on various web browsers (like Chrome, Firefox, and Safari) and
devices (such as desktops, tablets, and smartphones) to ensure a consistent user experience.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 9
 Software Testing
vi. Usability Testing: Usability testing assesses how easy and intuitive the software is for end-
users, focusing on user experience and satisfaction.
Example: In the library management system, usability testing might involve observing users as
they navigate the interface to borrow a book, noting any difficulties they encounter, such as
confusing layouts or unclear instructions, and gathering feedback to improve the overall user
experience.

Testing Process
The testing process typically involves several key stages to ensure software quality and functionality.
Here’s a structured overview of the testing process:
1. Requirement Analysis: Understand and analyze the requirements to define what needs to be tested.
 Activities:
o Review requirement documents, specifications, and user stories.
o Identify testable requirements and clarify ambiguities.
2. Test Planning: Outline the testing approach, resources, schedule, and scope.
 Activities:
o Define the testing strategy (manual, automated, etc.).
o Determine resource requirements (tools, team members).
o Create a test plan document outlining scope, objectives, timelines, and deliverables.
3. Test Case Design: Develop detailed test cases based on requirements and specifications.
 Activities:
o Write test cases, including steps, expected results, and test data.
o Review and validate test cases with stakeholders.
5. Test Execution: Execute the test cases and record the results.
 Activities:
o Run test cases and document actual results.
o Identify and log defects for any test case that fails.
o Retest fixed defects to verify corrections.
7. Defect Reporting and Management: Manage identified defects effectively.
 Activities:
o Log defects in a tracking system with relevant details (steps to reproduce, severity, etc.).
o Communicate defects to development teams and prioritize them for fixing.
8. Regression Testing: Ensure that new code changes do not adversely affect existing functionalities.
 Activities:
o Execute previously passed test cases to validate unchanged parts of the application.
o Automate regression tests where applicable for efficiency.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 10
 Software Testing

Cyclomatic complexity
 Independent path is an execution flow from the start point to the end point.
 Cyclomatic complexity is a software metric used to measure the complexity of a program by
quantifying the number of linearly independent paths through the source code. It is calculated
based on the control flow of the program, specifically focusing on decision points like if
statements, loops, and case statements.

Formula:
The Cyclomatic complexity (CC) can be calculated using the formula:

CC = E – N + 2

Where:

 E = Number of edges in the control flow graph

 N = Number of nodes in the control flow graph

Example: Consider the following flow graph

Number of Nodes = 13
Number of edges = 17

Cyclomatic Complexity, CC:

CC= E-N+2
= 17-13+2

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 11
 Software Testing
Why It Is Used:
 Code Reviews: During code reviews, developers can use cyclomatic complexity to identify
areas of the code that may be overly complex.
 Test Case Design: Cyclomatic complexity can help guide the design of test cases, ensuring
that all possible execution paths are covered.
 Identify Potential Risks: Code with high cyclomatic complexity is more prone to bugs and
defects, as it may be harder to follow the logic. Identifying such areas can help prioritize
refactoring or additional testing.

How software testing is differentiated from Quality Control, Quality

Assurance and Audit?
Software testing, quality control (QC), quality assurance (QA), and audits are all crucial components of
ensuring the quality of software systems, but they serve different purposes and are applied in distinct
contexts. Here's a breakdown of each term in the context of a Library Management System:

1. Software Testing
Software testing involves the process of executing a program or application to identify any bugs or
errors. It focuses on verifying that the software behaves as expected under various conditions.
Example: In a Library Management System, testing might involve:
 Functional Testing: Checking if users can successfully borrow and return books.
 Performance Testing: Assessing how the system performs under a heavy load (e.g., many users
searching for books simultaneously).
 Usability Testing: Evaluating whether the interface is user-friendly for both library staff and patrons.

2. Quality Control (QC)

Quality Control is a process by which entities review the quality of all factors involved in production. It
is typically product-oriented and focuses on identifying defects in the finished product.
Example: In the context of the Library Management System, QC might involve:
 Conducting systematic reviews of the system after the software testing phase to ensure that all identified
bugs have been resolved.
 Verifying that the system meets the specified requirements (e.g., accurate tracking of book inventory)
before it is deployed.

3. Quality Assurance (QA)

Quality Assurance is a proactive process that aims to improve and ensure the quality of the development
and production processes. It is focused on preventing defects through the implementation of processes
and best practices.
Example: For the Library Management System, QA might include:
 Establishing coding standards and guidelines that developers must follow during the software
development phase.
 Conducting regular training sessions for developers on best practices in software development to ensure
they understand how to write high-quality code.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 12
 Software Testing

4. Audit
An audit is a systematic examination of a process or system to ensure compliance with established
standards or regulations. It can be internal or external and often involves a review of the effectiveness
of quality processes.
Example: In the Library Management System context, an audit might involve:
 Reviewing the system’s documentation and processes to ensure compliance with regulatory standards
(e.g., data protection laws for patron information).
 Assessing whether the testing and QC processes were followed correctly and if the QA processes are
effectively implemented.

Summary
 Software Testing: Focuses on finding bugs and ensuring that the software works as intended.
 Quality Control (QC): Involves inspecting the final product to ensure it meets specified requirements.
 Quality Assurance (QA): Aims to improve the development process to prevent defects before they occur.
 Audit: Reviews compliance with standards and the effectiveness of QA/QC processes.

Prof. Dr. Engr. A.K.M. Masum, Dept. of Software Engineering, DIU Page Number- 13