Chapter-1 => Fundamentals of Testing

Verification: Confirmation by examination and through provision of objective evidence that specified
requirements have been fulfilled i.e. Have we build the system as per requirement specifications.

Validation: Confirmation by examination and through provision of objective evidence that the
requirements for a specific intended use or application have been fulfilled i.e. Have we build the right
system.

Test Objectives
- Evaluate work products like requirements, user stories, design and code by using static testing
techniques such as reviews.
- Verify all specified requirements have been fulfilled
- Validate the test object is complete and works as the users and stakeholders expect
- Build confidence in level of quality where highest risk flows pass and failures in other flows are
acceptable
- To prevent defects
- To find failures and defects
- Provide sufficient information to stakeholders to allow them to make decisions with entry and
exit criteria
- Reduce the level of risk
- To comply with legal, contractual, regulatory requirements and standards and verify for
compliance

During component testing:

- Objective is to achieve a given level of code coverage by component tests i.e. to assess how
much of code has actually been exercised by a set of tests and to add additional tests to exercise
parts of code that have not yet been covered/tested.
- Another objective is to find as many failures as possible so that the underlying defects are
identified and fixed as early as possible

During UAT:
- Objective is confirm the system works as expected (validation) and satisfies requirement
(verification)
- Another objective is to evaluate risk of releasing the system at a given time
During Maintenance testing:
- Objective is to perform regression testing
- Operational testing focussing on reliability, security and performance availability

Testing Contribution:

1
Quality Assurance: It is associated with ensuring that a company’s standard ways of performing various
tasks are carried out correctly not necessarily only related to testing. Such procedures are written in
quality handbook that everyone is suppose d to follow. If the process is carried out in the defined
processes, the output product will be of higher quality.
Root cause analysis and retrospectives are used to help in improving the processes for more effective
quality assurance. Also these standards are used to showcase the evidences in audit.

Quality Control: It is related to quality of products rather than processes i.e. to ensure that they have
achieved the desired level of quality with testing.

Error (mistake):
- A human action that produces an incorrect result
- Types of errors:
o Syntax error
o Logical error
o Semantic error
o Runtime error

2
 o Configuration error

Defect (Bug, Fault):

- An imperfection or deficiency in a work product where it does not meet its requirements or
specifications
- They can be caused by incomplete requirements, incorrect implementation or poor design
during development
- Types of defects:
o Requirement defects
o Design defects
o Coding defects
o Interface defects
o Performance defects

Bug:
- It refers to issue in software that causes it to behave unexpectedly
- They are found during testing lifecycle and can vary in severity
- They can be caused by variety of factors like incorrect logic, poor design or mistake during
coding process
- Types of bugs:
o Functional bugs- S/W does not perform its intended functionality as expected
o Performance bugs-
o Compatibility-
o Security-
o UI/UX- Errors or inconsistencies in the UI or experience, unclear navigation etc.

Fault:
- An unanticipated action
- It lies within the system and is caused by an error made during the development phase
- Types of faults:
o Logical fault
o Data fault

Failure:
- An event in which a component or system does not perform a required function within specified
limits
- This can be due to environment issue or network issue
- It impacts the end user and impact the business directly
- Types of failure:
o Functional failure
o Performance failure
o Security failure
o Reliability failure
o Integration failure

3
Aspect Bug Defect Error Fault Failure
Definition It is an informal A deviation Mistake made in state that The inability
name specified from specified the code; that's causes the of the
to the defect requirements. why we cannot software to fail software to
execute or to accomplish perform as
compile code. its essential expected.
function
Caused Incorrect logic, The Developer's A fault results Fault or
By syntax, or mismatch oversight or from an error environmental
implementation between misunderstanding in the system issues
actual and . triggering
expected incorrect
behavior behavior.
Raised By Test Eng. Testers Developers and Human End user
Automation eng. mistake
Detection Typically found Identified in Often detected Detected in Discovered
Phase during testing. testing or during testing under during
during development or specific integration or
validation. debugging. conditions production.
Examples A button click A login A loop with an off- Faulty API The system
does not feature fails by-one error. returning crashes
trigger an to meet incorrect data during a file
action. requirements. upload
operation.
Ways to Test-driven Using correct System review Confirm re-
prevent development. dev. and programming. testing.
Offer Techniques, Detect the issues Review the
programming peer reviews and prepare a requirements
language suitable mitigation and revisit the
support. plan. specifications.
Adjusting, Validate the fixes Implement
advanced, and and verify their current
operative quality and protective
development precision. techniques.
procedures. Categorize
Evaluating the and evaluate
code errors and
systematically issues.

4
Defects may or may not lead to failures, depending on inputs and other conditions. In some cases, a
defect can exist that will never cause a failure in actual use because the condition which can cause the
failure can never arise whereas a defect can exist that will not cause a failure during testing but may
result in failure in production.

Typical scenarios for Failure:

Requirement (BA) Design (Architect) Coding (Programmer) Product Quality

Correct Correct Correct Correct
Correct Correct Mistake Failure
Correct Mistake Mistake Failure
Mistake Mistake Mistake Failure

Multiplicative increase in cost to repair failure:

Root Cause:

A source of a defect such that if it is removed, the occurrence of the defect type is decreased or
removed. A root cause is generally organizational issue whereas defect is an individual action.

5
For the above example:

- Failure: Incorrect interest calculation for customers

- Defect: Wrong interest calculation in the code
- Root Cause: PO’s lack of knowledge about how interest should be calculated
- Effect: Customer complaint

Testing Principles:

1. Testing shows the presence if defects not their absence:

o No team, technique or test strategy can guarantee to achieve 100% defect-detection
percentage (DDP) or even 95% which is excellent
o Of course we reduce the likelihood of defects that remain undiscovered by adding more
scenarios or test cases but only an infinite number of tests can cut the risk down to zero
2. Exhaustive testing is impossible:

3. Early testing saves time and money:

6
4. Defects cluster together:

5. Beware of pesticide paradox:

6. Testing is context dependent:

7. Absence of errors is a fallacy

7
Test Process:

The factors that influence the particular test process include:

- SDLC model and project methodologies

- Test levels and test types being considered
- Product and project risks
- Business domain
- Operational constraints:
o Budget and resources
o Timescales and complexity
o Contractual and regulatory requirements
- Organizational policies and practices
- Required internal and external standards

Coverage: The degree to which specified coverage items have been determined to have been exercised
by a test suite expressed as percentage i.e. partial measure of the thoroughness of testing. Coverage is a
KPI (Key Performance Indicator) and help measure the achievement of test objectives

Test Basis: The body of knowledge used as the basis for test analysis and design i.e. requirement, user
stories, design or code

Test Activities: A test process consists of the following group of activities:

1. Test Planning:
a. Defining the objective of testing and approach for meeting those objectives within
project constraints and contexts.
b. Includes test techniques to use, tasks to be done, test schedule etc.
2. Test monitoring and control:

8
 a. Monitoring- Test management activity that involves checking the status of testing
activities, identifying any variances from the planned or expected status and reporting
status to stakeholders
b. Control- Test management task that deals with developing and applying a set of
corrective actions to get test project on track when monitoring shows deviation from
what was planned
c. One way to monitor test progress is by using exit criteria or definition of done in agile
development
3. Test analysis:
a. The activity that identifies test conditions by analysing the test basis i.e. what to test
including measurable coverage criteria
b. Test condition- An aspect of test basis that is relevant in order to achieve specific test
objectives.
4. Test design:
a. Activity of deriving and specifying test cases from test conditions.
b. Test analysis identifies things to test i.e. scenarios and test design makes these general
things specific for the component or system that we are testing
c. Test design include:
i. Design and prioritize test cases
ii. Identify test data
iii. Design test environment
iv. Capture bi-directional traceability between test basis, test conditions, test cases
and test procedures
5. Test implementation:
a. The activity that prepares the testware needed for test execution based on test analysis
and design
b. Test procedure: A sequence of test cases in execution order and any associated actions
that may be required to set up the initial preconditions and any wrap-up activities post
execution
c. In test implementation, we specify test procedures
d. In test design, we ask “how to test” and in test implementation we ask “do we have
everything to in place to run the tests”
e. Major activities:
i. Develop and prioritize the test procedures and potentially create automated
test scripts
ii. Arrange the test suite within a test execution schedule
iii. Verify the traceability
6. Test execution:
a. The process of running a test on the component or system under test producing actual
result
b. Testware: Work products produced during the test process for use in planning,
designing, executing, evaluating and reporting on testing
7. Test completion:

9
 a. Activity that makes test assets available for later use, leaves test environments in
satisfactory condition and communicates the results of testing to relevant stakeholders.
b. Activities include:
i. Check for all reported defects closed or moved to product backlog
ii. Create test summary report to be communicated to stakeholders
iii. Analyze lesson learned from completed testing activities
iv. Use the gathered information to improve test process maturity

Test Work Products

- Generic name given to any form of documentation, informal communication or artifacts that is
used in testing
- They are captured, stored and managed in configuration management tools/ test management
tools / defect management tools
- Test Planning work products: Test Plans
- Test monitoring & control work products: Different test reports
- Test analysis work products: Test conditions
- Test design work products: test cases
- Test implementation work product: test procedures, test suites, test execution schedule
- Test execution work product: documentation of the status of individual test cases or test
procedures, defect reports, documentation about which test items, test objects, test tools and
testware were involved in the testing
- Test completion work product: test summary report, action items for improving subsequent
iterations, change request or product backlog items, finalized testware

Traceability between the test basis and test work products:

10