Seunit 4
Seunit 4
Seunit 4
Software Testing is evaluation of the software against requirements gathered from users and
system specifications. Testing is conducted at the phase level in software development life cycle
or at module level in program code. Software testing comprises of Validation and Verification.
Software Validation
Validation is process of examining whether or not the software satisfies the user requirements. It
is carried out at the end of the SDLC. If the software matches requirements for which it was
made, it is validated.
Validation ensures the product under development is as per the user requirements.
Validation answers the question – "Are we developing the product which attempts all that
user needs from this software ?".
Validation emphasizes on user requirements.
Software Verification
Verification is the process of confirming if the software is meeting the business requirements,
and is developed adhering to the proper specifications and methodologies.
Errors - These are actual coding mistakes made by developers. In addition, there is a
difference in output of software and desired output, is considered as an error.
Fault - When error exists fault occurs. A fault, also known as a bug, is a result of an error
which can cause system to fail.
Failure - failure is said to be the inability of the system to perform the desired task.
Failure occurs when fault exists in the system.
Manual - This testing is performed without taking help of automated testing tools. The
software tester prepares test cases for different sections and levels of the code, executes
the tests and reports the result to the manager.
Manual testing is time and resource consuming. The tester needs to confirm whether or
not right test cases are used. Major portion of testing involves manual testing.
Automated This testing is a testing procedure done with aid of automated testing tools.
The limitations with manual testing can be overcome using automated test tools.
A test needs to check if a webpage can be opened in Internet Explorer. This can be easily done
with manual testing. But to check if the web-server can take the load of 1 million users, it is quite
impossible to test manually.
There are software and hardware tools which helps tester in conducting load testing, stress
testing, regression testing.
Testing Approaches
Functionality testing
Implementation testing
When functionality is being tested without taking the actual implementation in concern it is
known as black-box testing. The other side is known as white-box testing where not only
functionality is tested but the way it is implemented is also analyzed.
Exhaustive tests are the best-desired method for a perfect testing. Every single possible value in
the range of the input and output values is tested. It is not possible to test each and every value in
real world scenario if the range of values is large.
Black-box testing
It is carried out to test functionality of the program. It is also called ‘Behavioral’ testing. The
tester in this case, has a set of input values and respective desired results. On providing input, if
the output matches with the desired results, the program is tested ‘ok’, and problematic
otherwise.
In this testing method, the design and structure of the code are not known to the tester, and
testing engineers and end users conduct this test on the software.
Equivalence class - The input is divided into similar classes. If one element of a class
passes the test, it is assumed that all the class is passed.
Boundary values - The input is divided into higher and lower end values. If these values
pass the test, it is assumed that all values in between may pass too.
Cause-effect graphing - In both previous methods, only one input value at a time is
tested. Cause (input) – Effect (output) is a testing technique where combinations of input
values are tested in a systematic way.
Pair-wise Testing - The behavior of software depends on multiple parameters. In
pairwise testing, the multiple parameters are tested pair-wise for their different values.
State-based testing - The system changes state on provision of input. These systems are
tested based on their states and input.
White-box testing
It is conducted to test program and its implementation, in order to improve code efficiency or
structure. It is also known as ‘Structural’ testing.
In this testing method, the design and structure of the code are known to the tester. Programmers
of the code conduct this test on the code.
Control-flow testing - The purpose of the control-flow testing to set up test cases which
covers all statements and branch conditions. The branch conditions are tested for both
being true and false, so that all statements can be covered.
Data-flow testing - This testing technique emphasis to cover all the data variables
included in the program. It tests where the variables were declared and defined and where
they were used or changed.
Testing Levels
Testing itself may be defined at various levels of SDLC. The testing process runs parallel to
software development. Before jumping on the next stage, a stage is tested, validated and verified.
Testing separately is done just to make sure that there are no hidden bugs or issues left in the
software. Software is tested on various levels -
Unit Testing
While coding, the programmer performs some tests on that unit of program to know if it is error
free. Testing is performed under white-box testing approach. Unit testing helps developers
decide that individual units of the program are working as per requirement and are error free.
Integration Testing
Even if the units of software are working fine individually, there is a need to find out if the units
if integrated together would also work without errors. For example, argument passing and data
updation etc.
System Testing
The software is compiled as product and then it is tested as a whole. This can be accomplished
using one or more of the following tests:
Functionality testing - Tests all functionalities of the software against the requirement.
Performance testing - This test proves how efficient the software is. It tests the
effectiveness and average time taken by the software to do desired task. Performance
testing is done by means of load testing and stress testing where the software is put under
high user and data load under various environment conditions.
Security & Portability - These tests are done when the software is meant to work on
various platforms and accessed by number of persons.
Acceptance Testing
When the software is ready to hand over to the customer it has to go through last phase of testing
where it is tested for user-interaction and response. This is important because even if the
software matches all user requirements and if user does not like the way it appears or works, it
may be rejected.
Alpha testing - The team of developer themselves perform alpha testing by using the
system as if it is being used in work environment. They try to find out how user would
react to some action in software and how the system should respond to inputs.
Beta testing - After the software is tested internally, it is handed over to the users to use
it under their production environment only for testing purpose. This is not as yet the
delivered product. Developers expect that users at this stage will bring minute problems,
which were skipped to attend.
Regression Testing
Whenever a software product is updated with new code, feature or functionality, it is tested
thoroughly to detect if there is any negative impact of the added code. This is known as
regression testing.
Testing Documentation
Before Testing
Testing starts with test cases generation. Following documents are needed for reference –
The following documents may be required while testing is started and is being done:
Test Case document - This document contains list of tests required to be conducted. It
includes Unit test plan, Integration test plan, System test plan and Acceptance test plan.
Test description - This document is a detailed description of all test cases and
procedures to execute them.
Test case report - This document contains test case report as a result of the test.
Test logs - This document contains test logs for every test case report.
After Testing
Test summary - This test summary is collective analysis of all test reports and logs. It
summarizes and concludes if the software is ready to be launched. The software is
released under version control system if it is ready to launch.
We need to understand that software testing is different from software quality assurance,
software quality control and software auditing.
Functional testing: It is a type of software testing that validates the software systems against the
functional requirements. It is performed to check whether the application is working as per the
software’s functional requirements or not. Various types of functional testing are Unit testing,
Integration testing, System testing, Smoke testing, and so on.
Non-functional testing: It is a type of software testing that checks the application for non-
functional requirements like performance, scalability, portability, stress, etc. Various types of
non-functional testing are Performance testing, Stress testing, Usability Testing, and so on.
Maintenance testing: It is the process of changing, modifying, and updating the software to keep
up with the customer’s needs. It involves regression testing that verifies that recent changes to
the code have not adversely affected other previously working parts of the software.
Apart from the above classification software testing can be further divided into 2 more ways of
testing:
Manual testing: It includes testing software manually, i.e., without using any automation tool or
script. In this type, the tester takes over the role of an end-user and tests the software to identify
any unexpected behavior or bug. There are different stages for manual testing such as unit
testing, integration testing, system testing, and user acceptance testing. Testers use test plans, test
cases, or test scenarios to test software to ensure the completeness of testing. Manual testing also
includes exploratory testing, as testers explore the software to identify errors in it.
Automation testing: It is also known as Test Automation, is when the tester writes scripts and
uses another software to test the product. This process involves the automation of a manual
process. Automation Testing is used to re-run the test scenarios quickly and repeatedly, that were
performed manually in manual testing.
Apart from Regression testing, Automation testing is also used to test the application from a
load, performance, and stress point of view. It increases the test coverage, improves accuracy,
and saves time and money when compared to manual testing.
Black box Testing: Testing in which the tester doesn’t have access to the source code of the
software and is conducted at the software interface without any concern with the internal logical
structure of the software known as black-box testing.
White box Testing: Testing in which the tester is aware of the internal workings of the product,
has access to its source code, and is conducted by making sure that all internal operations are
performed according to the specifications is known as white box testing.
Grey Box Testing: Testing in which the testers should have knowledge of implementation,
however, they need not be experts.
Unit testing: It a level of the software testing process where individual units/components of a
software/system are tested. The purpose is to validate that each unit of the software performs as
designed.
Integration testing: It is a level of the software testing process where individual units are
combined and tested as a group. The purpose of this level of testing is to expose faults in the
interaction between integrated units.
System testing: It is a level of the software testing process where a complete, integrated
system/software is tested. The purpose of this test is to evaluate the system’s compliance with the
specified requirements.
Acceptance testing: It is a level of the software testing process where a system is tested for
acceptability. The purpose of this test is to evaluate the system’s compliance with the business
requirements and assess whether it is acceptable for delivery.
Continuous testing: Project teams test each build as it becomes available thus it enables software
to be validated in real environments earlier in the development cycle, reducing risks and
improving the functionality and design.
Involve users: It is very important for the developers to involve users in the process and open-
ended questions about the functionality required in the application. This will help to develop and
test the software from the customer’s perspective.
Divide tests into smaller parts: Dividing tests into smaller fractions save time and other resources
in environments where frequent testing needs to be conducted. This also helps teams to make
better analyses of the tests and the test results.
Metrics and Reporting: Reporting enables the team members to share goals and test results.
Advanced tools integrate the project metrics and present an integrated report in the dashboard
that can be easily reviewed by the team members to see the overall health of the project.
Don’t skip regression testing: Regression testing is one of the most important steps as it
encourages the validation of the application. Thus, it should not be skipped.
Programmers should avoid writing tests: Test cases are usually written before the start of the
coding phase so it is considered a best practice for programmers to avoid writing test cases as
they can be biased towards their code and the application.
Service virtualization: Service virtualization simulates the systems and services that are not yet
developed or are missing. Thus, enabling teams to reduce dependency and start the testing
process sooner. They can modify, and reuse the configuration to test different scenarios without
having to alter the original environment.
Product quality: Testing ensures the delivery of a high-quality product as the errors are
discovered and fixed early in the development cycle.
Customer satisfaction: Software testing aims to detect the errors or vulnerabilities in the software
early in the development phase so that the detected bugs can be fixed before the delivery of the
product. Usability testing is a type of software testing that checks the application for how easily
usable it is for the users to use the application.
Cost-effective: Testing any project on time helps to save money and time for the long term. If the
bugs are caught in the early phases of software testing, it costs less to fix those errors.
Security: Security testing is a type of software testing that is focused on testing the application
for security vulnerabilities from internal or external sources.
In this section, we are going to discuss the difference between Stubs and Drivers; and see a
brief introduction of them.
Stubs and drivers are two types of test harness that are a collection of software and test, which
means they are designed together to test a unit of a program by accelerating the change of
situations at the same time as regularly checking its performances and results.
In the Software Testing process, the Stubs and Drivers are those components that are used as a
short-term alternative for a module.
There are several elements in the Software Testing Life Cycle, which play an essential role in the
testing process to be more precise and trouble-free.
All the components connected to testing and try to enhance their quality, helps us to deliver an
exact and predictable outcome, and facilities the fulfilment of the specified specifications.
In Software Testing, the words stub and drivers described as a replica of the modules that
operate as an alternative to the new or missing modules.
Stubs are mainly used in top-down integration testing; on the other hand, drivers are mainly
used in bottom-up integration testing individually and designed the enhance the testing
process.
Both Stubs and drivers are the essential part of the basic software development and software
testing process. Thus, to help us understand the substance of stubs and drivers in software
testing, we will see an exhaustive discussion.
o A stub is a replica of a module that collects the data and develops many possible data.
However, it executes like an actual module and is mainly used to test modules.
o Generally, stubs are created by software developers in order to use them instead of
modules if the particular modules are miss or not yet developed.
o By using these test stubs, the test engineers can simulate the performance of the lower-
level modules, which are not yet joined with the software. Furthermore, it helps us to
accelerate the activity of the missing modules.
Types of Stubs
In the top-down approach of incremental integration testing, the stubs are divided into four
essential parts, which are as follows:
o Demonstrate the trace message.
o Exhibits the parameter values.
o Returns the consistent values, which are handled by the modules or the components.
o Returns the values of the specific parameters, which were utilized by testing components
or modules.
o The Drivers establish the test environments and takes care of the communication,
estimates results, and also sends the reports.
o These are just like stubs and used by software test engineers in order to accomplish the
missing or incomplete modules/ components requirements.
o The drivers are mainly developed in the Bottom-up approach of incremental
integration testing.
o Generally, drivers are bit complex as compared to the stubs.
o These can test the lower levels of the code when the upper-level modules or codes are not
developed or missing.
o In other words, we can say that the Drivers perform as pseudo-codes, which are mainly
used when the stub modules are completed; however, the initial modules/components are
not prepared.
The below facts explain the critical differences between stubs and drivers during the integration
testing process.
o Stubs and Drivers design as a dummy for the missing or inaccessible modules or the
components.
o Most commonly, stubs and drivers are used in the incremental integration testing,
where stubs are used in top-bottom methodology while drivers are used in a bottom-up
methodology.
o Usually, developers and unit test engineers are included in the creation of stubs and
drivers.
o Even though it provides an ease to perform the separate components or the
modules without worrying about the accessibility of other modules, and leads them into a
time-consuming process, as it involves the development of replica for all the missing
modules.
o Precisely, stubs and drivers are developed for each module holding different purposes.
Conclusion
To ensure the correctness and efficiency of testing, it is necessary to develop the stubs and
drivers that satisfy the requirements of unfinished modules.And perform as simulated
modules which are requested for testing the features of main modules or components.
They are computer programs that work as an alternative and replicate the functionalities of the
other modules, which help us simplify the software testing activities.
Alpha Testing is performed by the Testers within the organization whereas Beta Testing
is performed by the end users.
Alpha Testing is performed at Developer’s site whereas Beta Testing is performed at
Client’s location.
Reliability and Security testing are not performed in-depth in Alpha Testing while
Reliability, Security and Robustness are checked during Beta Testing.
Alpha Testing involves both Whitebox and Blackbox testing whereas Beta Testing
mainly involves Blackbox testing.
Alpha Testing requires testing environment while Beta Testing doesn’t require testing
environment.
Alpha Testing requires long execution cycle whereas Beta Testing requires only few
weeks of execution.
Critical issues and bugs are addressed and fixed immediately in Alpha Testing whereas
issues and bugs are collected from the end users and further implemented in Beta Testing.
Alpha Testing performed at Beta testing is performed at a client location or end user of
developer’s site the product
Reliability and Security Testing are Reliability, Security, Robustness are checked during Beta
not performed in-depth Alpha Testing Testing
Alpha testing is to ensure the quality Beta testing also concentrates on the quality of the product,
of the product before moving to Beta but gathers users input on the product and ensures that the
testing product is ready for real time users.
Traditional Beta Testing: Product is distributed to the target market, and related data is
gathered in all aspects. This data can be used for Product improvement.
Public Beta Testing: Product is publicly released to the outside world via online channels and
data can be gathered from anyone. Based on feedback, product improvements can be done. For
example, Microsoft conducted the largest of all Beta Tests for its OS — Windows 8 before
officially releasing it.
Technical Beta Testing: Product is released to the internal group of an organization and gathers
feedback/data from the employees of the organization.
Focused Beta: Product is released to the market for gathering feedback on specific features of
the program. For example, important functionality of the software.
Post-release Beta: Product is released to the market and data is gathered to make improvements
for the future release of the product.
Phases of Testing
Alpha and Beta tests are typically carried for “off-the-shelf” software or product-oriented
companies. The Phases of Testing for a product company typically varies from a service-oriented
organization. Following is the testing phase adopted by product firms
Pre-Alpha:- Software is a prototype. UI is complete. But not all features are completed. At this
stage, software is not published.
Alpha: Software is near its development and is internally tested for bugs/issues
Beta: Software is stable and is released to a limited user base. The goal is to get customer
feedback on the product and make changes in software accordingly
Release Candidate (RC): Based on the feedback of Beta Test, you make changes to the
software and want to test out the bug fixes. At this stage, you do not want to make radical
changes in functionality but just check for bugs. RC is also put out to the public
Note: Above is a standard definition of the Testing stages but in order to garner marketing buzz,
companies combine stages like “pre-alpha beta”, “pre-beta” etc.
Provides better view about the reliability of the software at an early stage
Helps simulate real time user behavior and environment.
Detect many showstopper or serious errors
Ability to provide early detection of errors with respect to design and functionality
Static testing
Static Testing Techniques, also known as dry run testing, may give you a feeling of
drowsiness, but don’t fall asleep yet! In technical words, this testing technique ensures the
software development process runs smoothly and without a hitch. This technique involves
analyzing the software documentation, code, and functional specifications. Yeah, it may
sound bland, but it’s a necessary evil. By doing a once-over of the documents and checking for coding
errors, typos, or inconsistencies, we can identify potential issues before they arise.
Static Testing is a valuable technique because it helps detect errors and defects in software before it is
tested, i.e., executed. This can save time, resources, and costs in the long run. Also, it catches issues
early when they are easier and less expensive to fix.
For example, You review software requirements as part of Static Testing. You may identify potential
conflicts that could be difficult to address later in the development cycle. Similarly, if you review code
during Static Testing, you may spot issues like
coding errors,
inconsistent naming conventions.
that could negatively impact the software’s performance or quality. By catching issues early on, teams
can identify areas for improvement and ensure that the software meets the required standards and
specifications. This can improve quality, increase user satisfaction, and reduce maintenance and support
costs over time.
Verify that the source code is complete and can be compiled & built.
Identify areas of the software that may have logical, structural, formatting, or syntactic errors.
Features:
Requirements & Specifications Review: This includes analyzing the functional and non-functional
requirements. We can also look at the feasibility of meeting the specified needs.
Structural Design Analysis: We must review the system’s static structure and flow diagrams to ensure it
meets the system requirements.
Code Review: This involves manually inspecting and analyzing the source code to ensure the absence of
errors and adherence to conventions and standards.
Database Analysis: We need to check for inconsistencies and ensure that all the database objects are
consistent and aligned with the database design.
Documentation Review: We must assess the documentation to ensure accuracy and readability.
User Manual Review: We must review the user guide to ensure users can easily use the system.
Review
Reviews are a necessary feature of Static Testing. It enables testers to identify defects and issues in
documentation, such as requirements and design. The importance of reviews lies in detecting the
sources of failure at the earliest stage.
This is a software testing technique involving analyzing code without executing it. Static Analysis aims to
uncover structural defects in the code developers write. Those can be such as
syntax violations,
Developers commonly use code analysis tools to analyze the code and identify issues. This can also help
detect structural issues in code and enforce coding standards.
Static Testing provides several benefits to software development projects. Here are some of the key
benefits:
Early defect detection: Static Testing helps detect errors, defects, and bugs early in the development
process when they are most cost-effective and easier to fix.
Improved code quality: Static Testing helps ensure that code is well-structured, easy to maintain, and
meets industry-recognized coding standards. This leads to higher code quality and reduces the chance of
bugs and defects in the final product.
Reduced costs and time: Detecting defects early on reduces the costs and time associated with fixing
them. This can save significant amounts of resources throughout the development cycle.
Prevention of common issues: Static Testing helps prevent common issues such as syntax errors, null
pointer exceptions, and resource leaks. By proactively addressing these issues during development,
teams can avoid problems later.
Improved collaboration: Static Testing helps identify issues in a collaborative environment, promoting
knowledge-sharing and fostering communication between team members.
Here are some of the most common manual methods of static testing:
Inspections: A formal review process where reviewers examine code and documentation for errors.
Inspections are typically led by a moderator and follow a defined checklist.
Walkthroughs: A less formal review process where a developer or team of developers presents their
work to others for feedback. Walkthroughs are an excellent way to get early feedback on code and
design.
Informal reviews: A casual review of code or documentation by another developer or team member.
Informal reviews can be a quick and effective way to catch errors.
Audits: A formal review of code and documentation to ensure they comply with specific standards or
regulations. Independent auditors typically perform audits.
Manual static testing methods can be very effective at identifying errors and defects in software.
However, they can also be time-consuming and expensive. To be most effective, manual static testing
should be used with other testing methods, such as dynamic and automated testing.
Advantages
1. Early detection of defects: Static testing allows for identifying defects in the early stages of
the software development life cycle. This helps reduce the cost and effort required to fix these defects
later.
2. Cost-effective: Static testing is a cost-effective method as it does not require the execution of code or
running test cases. It can be performed by simply reviewing documents, code, or designs, making it less
time-consuming and resource-intensive than dynamic testing.
3. Improved software quality: Static testing helps improve the software’s quality by identifying defects
early on. It ensures that potential issues are addressed before they impact the functionality or
performance of the system.
4. Facts sharing and collaboration: Static testing encourages knowledge sharing and collaboration
among team members. It allows developers, testers, and other stakeholders to review and discuss code
or documentation, leading to better understanding and improved communication within the team.
Disadvantages
1. Limited coverage: Static testing primarily focuses on the analysis of code and documentation, which
means it may not uncover all possible defects or issues that could arise during runtime. It does not
involve the execution of the software, so it cannot identify dynamic errors or bugs that occur only when
the program is running.
2. Time-consuming: Static testing can be time-consuming, especially for large and complex software
systems. Reviewing code and documentation line by line requires significant effort and attention to
detail, which can slow down the overall development process.
3. False sense of security: Relying solely on static testing may give developers a false sense of security
about the quality of their software. While it helps identify certain defects, it cannot guarantee that all
potential issues have been resolved. Dynamic testing methods are still necessary to validate the
behavior and performance of the software in real-world scenarios.
Summary
To sum up everything about static testing, we can say it is a way of testing software without running it.
The idea is to find errors early in the development process by looking at the code, documentation, and
other related things. This approach helps to save costs and improve the quality of the software. The
testing may only find simple errors, and it has a chance of overlooking certain types of problems. To
make this testing effective, selecting the appropriate techniques, involving the right people, and doing
reviews regularly are essential. Combining static testing with other testing methods helps to achieve a
better quality product.