CSE 3421

Software Testing

SUMMER 2021
MD. RAFI-UR-RASHID
LECTURER, DEPT. OF CSE, UIU
Why This Lesson?
No customer likes to buy faulty products.
Same goes for the software industry
What is software testing?

Software testing is the process of analyzing a software

item to detect the differences between existing and
expected conditions (i.e., bugs)
When should we test software?

Answer: Throughout the

whole development
process
Testing Models
Black box testing

Ignores the internal mechanism of a system or component and

focuses solely on the outputs generated in response to selected
inputs and execution conditions.
Black box testing
 Interface visible, internals unknown

 From the outside, you are testing its functionality against the specs

 For software this is testing the interface

 What is input to the system?
 What you can do from the outside to change the system?
 What is output from the system?

 Tests the functionality of the system by observing its external

behavior
Advantages
 Robust with respect to changes in implementation
- Test data need not be changed when code is changed

 Allows for independent testers

- Testers need not be familiar with code

 Less time-consuming
Disadvantages
 It will miss bugs in the implementation that are not covered
by the specification

 Does not allow performance optimization and alternate

algorithms
White box testing

takes into account the internal mechanisms of a system

or component.
White box testing
 Given knowledge of the internal workings, you thoroughly test what
is happening on the inside

 Close examination of procedural level of detail

 Logical paths through code are tested

- Conditionals
- Loops
- Branches

 Status is examined in terms of expected behaviors

Advantages
 Allows thorough testing of the software

 Helps finding hidden bugs & errors

 Yields useful test cases.

 Enables code optimization and experiment with

alternate algorithms.
Disadvantages
 Impossible to thoroughly exercise all paths
- Exhaustive testing grows without bound

 Requires Code Access

 Skilled tester required

- Requires high level knowledge of internals of the software under test.

 More time-consuming
Functional testing
 FUNCTIONAL TESTING is a type of software testing whereby the system is
tested against the functional requirements specifications (FRS).

 Functions (or features) are tested by feeding them input and examining the
output.

 This type of testing is not concerned with how processing occurs, but rather,
with the results of processing.

 During functional testing, Black Box Testing technique is used in which the
internal logic of the system being tested is not known to the tester.
Non-Functional/ Technical testing
 Technical TESTING is a type of software to check non-functional aspects
(performance, usability, reliability, security etc.) of a software.

 It is explicitly designed to test the readiness of a system as per nonfunctional

parameters which are never addressed by functional testing

 It basically uses white-box testing, sometimes black box

Types of Testing
Unit testing
 Testing of individual software or hardware units or groups of related
units
 Done by programmer(s)
 Generally white box
 Verify that code does what it is intended to do at a very low
structural level
 Automation desirable for repeatability
 Component Testing: Almost identical to unit testing, except at a
slightly larger scale
Integration testing
 Testing in which software components, hardware components, or both are
combined and tested to evaluate the interaction between them
 Done by programmer as they integrate their code into code base
 Verifies that units work together when they are integrated into a larger code
base
 Just because the components work individually, that does not mean that they all
work together when integrated
 Generally white box, maybe some black box
 Automation desirable for repeatability
System Testing

 System testing is testing conducted on a complete integrated

system to evaluate the system's compliance with its specified
requirements.
 System testing is performed on the entire system in the context of a
Functional Requirement Specification(s) (FRS) and/or a Non-
functional/Technical Requirement Specification that in combination
generate Software Requirement Specification (SRS).
 So, it covers non-functional requirements as well.
Load, and Stress Testing

 Load testing is a kind of Performance Testing which determines a

system's performance under real-life load conditions. Load testing
usually identifies the maximum operating capacity of an
application.
 Load can be the expected concurrent number of users on the
application performing a specific number of transactions within the
set duration. This test will give out the response times of all the
important business critical transactions.

 Stress testing implies evaluating a system beyond the limits of its

specification. In Stress testing, we measure the breakpoint of a
system.
Difference between load and stress
test
 The goal of a load test is to prove that a system can handle the
expected volume with minimal to acceptable performance
degradation.
 A stress test is a test designed to increase the number of
simultaneous requests on a system past a point where performance
is degraded, possibly even to the point of complete failure.
 Difference in short: If you are testing normal, expected load, this is
load testing. But when you want to determine how the system
behaves under extreme load (DoS) and when it breaks, this is stress
testing.
Security testing

 SECURITY TESTING is a type of software testing that intends to

uncover vulnerabilities of the system and determine that its data
and resources are protected from possible intruders.
 Vulnerability Scanning: This is done through automated software to scan
a system against known vulnerability signatures.
 Ethical hacking: It's hacking an Organization Software systems. Unlike
malicious hackers ,who steal for their own gains , the intent is to expose
security flaws in the system.
 Penetration testing: This kind of testing simulates an attack from a
malicious hacker. This testing involves analysis of a particular system to
check for potential vulnerabilities to an external hacking attempt.
Regression testing
 Regression testing is selective retesting of a system or component
to verify that modifications have not caused unintended effects
and that the system or component still complies with its specified
requirements
 Subset of the original set of test cases.
 Core group of tests re-run often after any significant changes
 Choose a representative sample of tests that exercise all the existing
functionalities
 Chose additional test cases that are most likely to be affected by
the change
 Smoke test: a subset of the regression test cases that establish
that the system is stable and all major functionality is present and
works under “normal” conditions
Beta testing
 Organization can offer an advance partial or full version of a software package
free to one or more potential users.
 Users use the software as they wish, with the understanding that they will report any
errors revealed during usage back to the organization.

 Advantages
 Identification of unexpected errors
 Low costs
 Wider population / environment

 Disadvantages
 Lack of systematic testing
 Low quality error reports
Acceptance testing

 Formal testing conducted to determine whether or not a

system satisfies its acceptance criteria (the criteria the
system must satisfy to be accepted by a customer) and to
enable the customer to determine whether or not to
accept the system
 Generally done by customer/customer representative in
their environment through the GUI
 Definitely black box
 Testing Hierarchy

Beta testing

Regression
testing
Static & Dynamic testing

 Static testing refers to testing something that’s not running.

 It is sort of examining and reviewing the work. E.g. code
review, software inspection are some static testing
methods.
 Dynamic Testing involves working with the software, giving
input values and checking if the output is as expected.
 Unit Tests, Integration Tests, System Tests and Acceptance
Tests are few of the Dynamic Testing methodologies.
Code smell
 Code smell implies code components that are absolute violations of the
fundamentals of developing software that decrease the quality of code.
 Has negative impact on readability and maintainability.
 Code smells are usually not bugs; they are not technically incorrect and
do not prevent the program from functioning.
 Instead, they indicate weaknesses in design that may
slow down development or increase the risk of bugs
or failures in the future.
 For instance Long function, code duplication,
mysterious names etc.
 Tools such as CodeGrip, Checkstyle, PMD, FindBugs,
and SonarQube can automatically identify code smells.
Code Review
Code review
 Code review is the process of analyzing code written by another
developer on the project to judge whether it is of sufficient quality to
be integrated into the main project codebase.
 The formal variant of peer code review, which is better known as
software inspection or Fagan-inspection, has been an effective
quality improvement practice for a long time.
 Even with the benefits offered by software inspections, their
relatively high cost and formal requirements have reduced the
prevalence with which software teams adopt them.
 Hence, lightweight, informal, and tool-based code review practices
are being popular.
Code Review Efforts 32

Avg. 6 hours / week

10%~15% of developer
times

: 50,000 developers X 6 hours/ week X $50 /hour

= $15,000,000 / Week
Code Review Workflow
Author
Writes Edits Abandon
Code Code

Review
Request No

… Accepted?

Merge
Accept
Yes
Reviewer

Reviews Reviews Approval

Code Code
Simplified code review workflow
Main project
Code review repository branch
Pull to local branch Push to main

Reviews
Code
Yes
Developer’s Notifies
Local Push to Gerrit/github
Branch

Approved

Reviewer
Notifies developer No

Developer
Code Review Tools
Gerrit: https://code.google.com/p/gerrit/

Review Board: https://www.reviewboard.org/

Phabricator: https://phabricator.org/

Crucible: https://www.atlassian.com/software/crucible
Issues identified during code reviews

✓ Coding style ✓ Redundant checks ✓ Project design

violation

✓ Misunderstood ✓ Critical security ✓ Malicious code

requirements defects

✓ Inadequate ✓ Lack of exception

input validation ✓ Unsafe methods
handling
Practical Impacts
 Peer review can put strain on interpersonal team relationships.

 It´s difficult to have every piece of work critiqued by peers and to have
management evaluating and measuring defect density in your code.

 It´s extremely important that mangers create a culture of collaboration

and learning in peer review.

 While it´s easy to see defects as purely negative, each bug is actually
an opportunity for the team to improve code quality.
Other Benefits of Code review 38

 Project awareness

 Peer impression formation

 Consistent coding style

 Knowledge dissemination
Thank You