System Testing (I)
System Testing (I)
Software Testing
➢Strategic Approach to Software Testing,
➢ Testing Fundamentals Test Plan,
➢ Test Design,
➢Test Execution,
➢ Reviews,
➢ Inspection Auditing,
➢Alpha and Beta Testing of Products.
A. Objective: It describes the aim of the test plan, whatever the good
process and procedure they are going to follow in order to give quality
software to customers. The overall objective of the test is to find as many
defects as possible and to make software bug free. The test objective must
be broken into components and sub-components. In every component
following activities should be performed.
• List all the functionality, performance to be tested.
• Make goals and targets based on the application feature.
B. Test Strategy: It is a crucial document that is to be performed and
usually designed by the Test Manager. It helps to determine Test Effort and
Test cost. Test strategy helps to determine the features that are going to be
tested and the features that will not be tested. The scope can be divided
into two parts:
• In-Scope: The modules that are to be tested rigorously.
• Out Scope: The modules that are not to be tested rigorously.
Example: In an application A, B, C, D features have to be developed, but the
B feature has already been designed by other companies. So the
development team will purchase B from that company and perform only
integrated testing with A, B, C.
C. Testing Methodology: The methods that are going to be used for testing
depend on application to application. The testing methodology is decided
based on the feature and application requirements.
Since the testing terms are not standard, one should define what kind of
testing will be used in the testing methodology. So that everyone can
understand it.
D. Approach: The approach of testing different software is different. It deals
with the flow of applications for future references. It has two aspects:
• High-Level Scenarios: For testing critical features high-level
scenarios are written. For Example, login to a website, booking
from a website.
• The Flow Graph: It is used when one wants to make benefits such
as converging and merging easy.
E. Assumptions: In this phase, certain assumptions will be made.
Example:
• The testing team will get proper support from the development
team.
• The tester will get proper knowledge transfer from the
development team.
• Proper resource allocation will be given by the company to the
testing department.
F. Risk: All the risks that can happen if the assumption is breaking. For
Example, in the case of wrong budget estimation, the cost may overrun.
Some reason that may lead to risk is:
• Test Manager has poor management skills.
• Hard to complete the project on time.
• Lack of cooperation.
G. Backup/Mitigation Plan- If any risk is involved then the company must
have a backup plan, the purpose is to avoid errors. Some points to
resolve/avoid risk:
• Test priority is to be set for each test activity.
• Managers should have leadership skills.
• Training course for the testers.
H. Roles and Responsibilities: All the responsibilities and role of every
member in a particular testing team has to be recorded.
Example:
• Test Manager: Manages the project, takes an appropriate resource
and gives project direction.
• Tester: Identify the testing technique, verify the test approach, and
save project cost.
I. Scheduling: Under this, it will record the start and the end date of each
and every testing-related activity. For Example, writing test case date and
ending test case date.
J. Defect Tracking: It is an important process in software engineering as lots
of issue arises when you develop a critical system for business. If there is
any defect found while testing and that defect must be given to the
developer team. There are the following methods for the process of defect
tracking:
• Information Capture: In this, we take basic information to begin
the process.
• Prioritize: The task is prioritized based on severity and importance.
• Communicate: Communication between the identifier of bug and
fixer of bug.
• Environment: Test the application based on hardware and
software.
Example: The bug can be identified using bug tracking tools such as Jira,
Mantis, Trac.
K. Test Environment- It is the environment which the testing team will use
i.e. the list of hardware and software, while testing the application, the
things which are said to be tested will be written under this section. The
installation of software is also checked under this.
Example:
• Software configuration on different operating systems, such as
Windows, Linux, Mac, etc.
• Hardware Configuration depends on RAM, ROM, etc.
L. Entry and Exit Criteria: The set of conditions that should be met in order
to start any new type of testing or to end any kind of testing.
Entry Condition:
• Necessary resources must be ready.
• The application must be prepared.
• Test data should be ready.
Exit Condition:
• There should not be any major bug.
• Most test cases should be passed.
• When all test cases are executed.
Example: If the team member report 45% of the test cases failed, then
testing will be suspended until the developer team fixes all defects.
TEST DESIGN
Requirement Analysis
Test Planning
Test Execution
Test Execution is the process of executing the tests written by the tester to
check whether the developed code or functions or modules are providing
the expected result as per the client requirement or business requirement.
Test Execution comes under one of the phases of the Software Testing Life
Cycle (STLC).
Importance of Test Execution:
• The project runs efficiently: Test execution ensures that the
project runs smoothly and efficiently.
• Application competency: It also helps to make sure the
application’s competency in the global market.
• Requirements are correctly collected: Test executions make sure
that the requirements are collected correctly and incorporated
correctly in design and architecture.
• Application built in accordance with requirements: It also checks
whether the software application is built in accordance with the
requirements or not.
The following are the 5 main activities that should be carried out during the
test execution.
1. Defect Finding and Reporting: Defect finding is the process of
identifying the bugs or errors raised while executing the test cases
on the developed code or modules. If any error appears or any of
the test cases failed then it will be recorded and the same will be
reported to the respective development team. Sometimes, during
the user acceptance testing also end users may find the error and
report it to the team. All the recorded details will be reported to
the respective team and they will work on the recorded errors or
bugs.
2. Defect Mapping: After the error has been detected and reported to
the development team, the development team will work on those
errors and fix them as per the requirement. Once the development
team has done its job, the tester team will again map the test
cases or test scripts to that developed module or code to run the
entire tests to ensure the correct output.
3. Re-Testing: From the name itself, we can easily understand that
Re-Testing is the process of testing the modules or entire product
again to ensure the smooth release of the module or product. In
some cases, the new module or functionality will be developed
after the product release. In this case, all the modules will be re-
tested for a smooth release. So that it cannot cause any other
defects after the release of the product or application.
4. Regression Testing: Regression Testing is software testing that
ensures that the newly made changes to the code or newly
developed modules or functions should not affect the normal
processing of the application or product.
5. System Integration Testing: System Integration Testing is a type
of testing technique that will be used to check the entire
component or modules of the system in a single run. It ensures
that the whole system will be checked in a single test environment
instead of checking each module or function separately.
The test Execution technique consists of three different phases which will
be carried out to process the test result and ensure the correctness of the
required results. In each phase, various activities or work will be carried out
by various team members. The three main phases of test execution are the
creation of test cases, test case execution, and validation of test results. Let
us discuss each phase.
1. Creation of Test Cases: The first phase is to create suitable test cases for
each module or function. Here, the tester with good domain knowledge
must be required to create suitable test cases. It is always preferable to
create simple test cases and the creation of test cases should not be
delayed else it will cause excess time to release the product. The created
test cases should not be repeated again. It should cover all the possible
scenarios raised in the application.
2. Test Cases Execution: After test cases have been created, execution of
test cases will take place. Here, the Quality Analyst team will either do
automated or manual testing depending upon the test case scenario. It is
always preferable to do both automated as well as manual testing to have
100% assurance of correctness. The selection of testing tools is also
important to execute the test cases.
3. Validating Test Results: After executing the test cases, note down the
results of each test case in a separate file or report. Check whether the
executed test cases achieved the expected result and record the time
required to complete each test case i.e., measure the performance of each
test case. If any of the test cases is failed or not satisfied the condition then
report it to the development team for validating the code.
Testers can choose from the below list of preferred methods to carry out
test execution:
1. Run test cases: It is a simple and easiest approach to run test
cases on the local machine and it can be coupled with other
artifacts like test plans, test suites, test environments, etc.
2. Run test suites: A test suite is a collection of manual and
automated test cases and the test cases can be executed
sequentially or in parallel. Sequential execution is useful in cases
where the result of the last test case depends on the success of
the current test case.
3. Run test case execution and test suite execution
records: Recording test case execution and test suite execution is a
key activity in the test process and helps to reduce errors, making
the testing process more efficient.
4. Generate test results without execution: Generating test results
from non-executed test cases can be helpful in achieving
comprehensive test coverage.
5. Modify execution variables: Execution variables can be modified in
the test scripts for particular test runs.
6. Run automated and manual tests: Test execution can be done
manually or can be automated.
7. Schedule test artifacts: Test artifacts include video, screenshots,
data reports, etc. These are very helpful as they document the
results of the past test execution and provide information about
what needs to be done in future test execution.
8. Defect tracking: Without defect tracking test execution is not
possible, as during testing one should be able to track the defects
and identify what when wrong and where.
Test Execution Priorities are nothing but prioritizing the test cases
depending upon several factors. It means that it executes the test cases
with high efficient first than the other test cases. It depends upon various
factors. Let us discuss some of the factors to be considered while prioritizing
the test cases.
• Complexity: The complexity of the test cases can be determined
by including several factors such as boundary values of test cases,
features or components of test cases, data entry of test cases, and
how much the test cases cover the given business problem.
• Risk Covered: How much risk that a certain test case may undergo
to achieve the result. Risk in the form of time required to complete
the test case process, space complexity whether it is executed in
the given memory space, etc.,
• Platforms Covered: It simply tells that in which platform or
operating system the test cases have been executed i.e., test cases
executed in the Windows OS, Mac OS, Mobile OS, etc.,
• Depth: It covers how depth the given test cases cover each
functionality or module in the application i.e., how much a given
test procedure covers all the possible conditions in a single
functionality or module.
• Breadth: It covers how the breadth of the given test cases covers
the entire functionality or modules in the application i.e., how much
a given test procedure covers all the possible conditions in the
entire functionality or modules in the product or application.
The tester or the Quality Analyst team reports or notices the result of each
test case and records it in their documentation or file. There are various
results raised when executing the test cases. They are
• Pass: It tells that the test cases executed for the module or
function are successful.
• Fail: It tells that the test cases executed for the module or function
are not successful and resulted in different outputs.
• Not Run: It tells that the test cases are yet to be executed.
• Partially Executed: It tells that only a certain number of test cases
are passed and others aren’t met the given requirement.
• Inconclusive: It tells that the test cases are executed but it requires
further analysis before the final submission.
• In Progress: It tells that the test cases are currently executed.
• Unexpected Result: It tells that all the test cases are executed
successfully but provide different unexpected results.
The Test Execution Report is nothing but a document that contains all the
information about the test execution process. It is documentation that will
be recorded and updated by the QA team. In that, they just record all the
processes happening in the day-to-day test execution activities. The test
execution activities are nothing but executing the task related to testing.
The documentation or the report contains various information. They are:
• Who all are going to execute the test cases?
• Who is doing the unit testing, integration testing, system testing,
etc.,
• Who is going to write test cases?
• The number of test cases executed successfully.
• The number of test cases failed during the testing.
• The number of test cases executed today.
• The number of test cases yet to be executed.
• What are the automation test tools used for today’s test
execution?
• What are the modules/functions testing today?
• Recording the issues while executing the test cases.
• What is today’s testing plan?
• What is tomorrow’s testing plan?
• Recording the pending plans.
• Overall success rate.
• Overall failure rate.
These are the headings in the Test Execution Report:
• Test Summary Report Identifier.
• Summary.
• Variances.
• Comprehensive Assessment.
• Summary of Results.
• Evaluation.
• Summary of Activities.
• Approval.
Guidelines for Test Execution
• Write the suitable test cases for each module of the function.
• Assign suitable test cases to respective modules or functions.
• Execute both manual testing as well as automated testing for
successful results.
• Choose a suitable automated tool for testing the application.
• Choose the correct test environment setup.
• Note down the execution status of each test case and note down
the time taken by the system to complete the test cases.
• Report all the success status and the failure status to the
development team or to the respective team regularly.
• Track the test status again for the already failed test cases and
report it to the team.
• Highly Skilled Testers are required to perform the testing with less
or zero failures/defects.
• Continuous testing is required until success test report is
achieved.
Software Review
Software Review isa systematic inspection of
software by one or more individuals who work
together to find and resolve errors and defects in the
software during the early stages of the Software Development Life Cycle
(SDLC).
A software review is an essential part of the Software Development Life Cycle
(SDLC) that helps software engineers in validating the quality, functionality,
and other vital features and components of the software. It is a whole process
that includes testing the software product and it makes sure that it meets the
requirements stated by the client.
Usually performed manually, software review is used to verify various
documents like requirements, system designs, codes, test plans and test cases.
• (iii) Walkthrough:
Members of the development team is guided by author and
other interested parties and the participants ask questions
and make comments about defects.
• (v) Inspection:
In inspection the reviewers follow a well-defined process to
find defects.
Inspection
The term software inspection was developed by IBM in the early 1970s
when it was noticed that the testing was not enough sufficient to attain
high-quality software for large applications.
There are several ways that software inspection can improve software
quality:
Software Audit
➢ A software audit is a thorough review of a software
product to check its quality, progress, standards,
and regulations.
➢ It checks the health of a product and ensures that everything is going as
planned.
➢ It can be done by an internal team or any external independent auditors.
➢ If the audit is performed by an external auditor, it can add some financial
strain to the company and disrupt development to accommodate such audits.
➢ It is recommended to perform regular internal audits on the software to ensure
that proper regulations are followed and all licenses are up-to-date as it can
also save the company from unnecessary legal issues.
• Compliance audit: This audit checks if the process is within the given standards. If the
testing process has certain standards to adhere to, this audit ensures that it’s followed
• Process improvement: If there are any changes needed for the existing process, this
audit helps in identifying them. This is done by evaluating the various steps in the
process and identifying any problems, and eliminating them.
• Root cause analysis: This audit helps to find the root cause for a problem using
different testing processes. It is done for specific problems that require some attention
and need to be resolved.
Key differences between inspections and audits:
1. Purpose and Focus:
o Inspections primarily focus on identifying safety hazards in the
workplace. They revolve around people, places, and things.
o Audits, on the other hand, concentrate on evaluating
the processes adopted by an organization to prevent identified hazards
and address worker safety issues. Audits center on operations,
processes, and programs.
2. Frequency and Duration:
o Inspections usually occur monthly or at regular intervals. They are
relatively quick and might only take five or ten minutes at the start of
each shift.
o Audits are more in-depth and frequent. They can take several days to
complete, ensuring a thorough evaluation of compliance obligations.
3. Compliance Obligations:
o Inspections are typically something that a site is required to do by
a compliance obligation.
o Audits involve checking whether compliance obligations have been
met, including verifying that the required inspections have been
conducted.
o Inspections are the “do” part, while audits are the “check” part. Both are
essential tools for ensuring organizations operate in a manner that
complies with applicable laws and regulations.
Alpha Testing
Alpha Testing is a type of software testing performed to identify bugs
before releasing the product to real users or to the public. Alpha Testing
is one of the user acceptance tests. It is the first stage of software
testing, during which the internal development team tests the program
before making it available to clients or people outside the company.
Key points to remember:
• Work Done by Developers: The internal development team,
which consists of developers and testers, usually conducts
alpha testing in a controlled setting.
• Goal: Finding and fixing bugs, flaws, and usability issues is the
main goal before releasing the product for external users or
wider testing.
• Little User Engagement: Alpha testers are few in number and
frequently comprise members of the development team or those
intimately connected to the project.
• Environment: Alpha testing is typically carried out in a
development environment or laboratory that resembles actual
settings.
Beta Testing
Beta Testing is performed by real users of the software application in a real
environment. Beta testing is one type of User Acceptance Testing. A pre-
release version of the product is made available for testing to a chosen
set of external users or customers during the second phase of software
testing.
Key Points to remember:
• Actions Taken by Users: Customers or other users outside the
development team participate in beta testing. The software is
available to these users prior to its official release.
• Goal: The primary objective is to get input from actual users in
order to find any bugs, usability difficulties, or areas that need to
be improved before the product is formally released.
• Greater User Participation: In order to capture a variety of
viewpoints, a larger number of users—including a varied range,
can serve as beta testers.
• Environment: Real-world settings are used for beta testing to
simulate how users will interact with the program while
performing daily duties.
Difference between Alpha and Beta Testing
The difference between Alpha and Beta Testing is as follows:
Alpha Testing Beta Testing
Parameters
Alpha testing is
Beta testing is
performed by testers
performed by clients
who are usually
who are not part of the
internal employees of
organization.
the organization.
Performed by