Software Engineering

Requirement Engineering

NATIONAL UNIVERSITY OF
Dr. Syeda Javaria Imtiaz CS 3009 - Software Engineering COMPUTING AND EMERGING
SCIENCES
Testing Objectives
What is being tested and why?
Testing is a method used to determine whether each of the project’s
software deliverables meet the defined quality standards established for the
software development project.

Goal is production-ready software.

Testing
• There is a massive misunderstanding about testing: that it improves
software. It doesn't!
• Weighing yourself doesn't reduce your weight
• Going to the doctor doesn't make you healthy.

• Those things help to identify problems that you might choose to

resolve. Testing does too.

• The testing does not make the product better, even though it's part of a
process that does make a product better.

4
Writing test cases
• First you must understand the language fundamentals
• Sizes and limits of variables, platform specific information

• Second, you must understand the domain

• Read the requirements

• Think like a user – what possible things do they want to do

• Think about possible “mistakes”; i.e. Invalid input

• Think about impossible conditions or input

• What is the testing intended to prove?

• Correct operation – gives correct behavior for correct input
• Robustness – responds to incorrect or invalid input with proper results
• User acceptance – typical user behavior

• Write down the test cases

5
Writing Good Test Cases
• Test Cases need to be simple and transparent

• Create Test Case with end user in mind

• Avoid test case repetition

• Do not Assume
• Stick to the Specification Documents.

• Ensure 100% Coverage

• Test Cases must be identifiable.

• Implement Testing Techniques

• It's not possible to check every possible condition in your software application
• Testing techniques help you select a few test cases with the maximum possibility of finding a defect

6
Writing Good Test Cases
• Boundary Value Analysis (BVA)
• testing of boundaries for specified range of values.

• Repeatable and self-standing

• The test case should generate the same results every time no matter who tests
it

7
Writing a test case
While drafting a test case do include the following information:
• The description of what requirement is being tested
• Inputs and outputs or actions and expected results
• Test case must have an expected result.

• Verify the results are correct

 Testing Normal Conditions
 Testing Unexpected Conditions
 Bad (Illegal) Input Values
 Boundary Conditions

8
Writing test cases
• Cover all possible valid input
• Try multiple sets of values, not just one set of values
• Permutations of values

• Check boundary conditions

• Check for off-by-one conditions

• Check invalid input

• Illegal sets of value
• Wrong format: not integer
• Negative numbers
• Illegal input
• Impossible conditions: {2,3,8}, {2,3,5} {definition of a triangle}
• Totally bad input
• Text vs. Numbers, etc.

9
Writing test cases
• Beware of problems with comparisons
• How to compare two floating numbers
• Never do the following:
float a, b;
...
if (a == b)
• Is it 4.0000000 or 3.9999999 or 4.0000001 ?
• What is your limit of accuracy?

• In object oriented languages make sure whether you are comparing the contents of an
object or the reference to an object
String a = “Hello world!\n”
String b = “Hello world!\n”
if ( a == b )
vs.
if ( a.equals(b) )

10
Tips for testing
• You cannot test every possible input, parameter value, etc.
• So you must think of a limited set of tests likely to expose bugs.

• Think about boundary cases

• positive; zero; negative numbers; infinity; very small
• right at the edge of an array or collection's size (plus or minus one)

• Think about empty cases and error cases

• 0, -1, null; an empty list or array

• test behavior in combination

• maybe add usually works, but fails after you call remove
• make multiple calls; maybe size fails the second time only

11
Verification vs validation
• Verification:
− "Are we building the product right”.
− The software should conform to its specification.
− Defect detection and correction
− Comparison between implementation and the corresponding
specification

• Validation:
− "Are we building the right product”.
− The software should do what the user really requires.
− Defect prevention
− SW is traceable to customer requirements
 Example
Example of verification in the calculator application:

• Requirement: The calculator application should correctly add two numbers.

• Verification: You review the code for the addition operation and ensure that it correctly adds two
numbers by performing unit tests. You verify that the addition function returns the correct sum for
different input values, such as 2 + 2 = 4, 5 + 3 = 8, etc. This process confirms that the implemented
code behaves as expected according to the specified requirement.

Example of validation in the calculator application:

• User Need: Users expect the calculator application to accurately perform mathematical calculations
and provide correct results.
• Validation: You conduct user acceptance testing (UAT) with a group of users who perform various
calculations using the application. Users input different sets of numbers and verify that the calculator
produces accurate results for addition, subtraction, multiplication, and division operations. The
feedback from users confirms that the calculator application meets their needs and performs as
expected in real-world usage scenarios.
 Types/Strategies of testing
• Code Inspections

• Software (Module) Testing

 Unit Testing
 Functional Testing

• Integration Testing
 Compliance
 Interoperability Testing

• System Testing
 Recovery
 Security
 Environment

• Acceptance Testing and Release Testing Regression Testing

• Stress, Security, performance, Load Testing etc

14
 Three Levels of Correctness

• Possible correctness

 Obtaining correct output for some arbitrary input (single set).

 If the outcome is wrong, the program cannot possibly be correct.

 For example a multiply program for input 2 and 3, if result is 6; is

possibly correct.
 Three Levels of Correctness

Probable correctness

 Obtaining correct output for a number of carefully selected inputs.

 If all potential problematic areas are checked in this way, the program
is probably correct.
 Try several values, including the obvious “problem” values such as zero,
largest negative, 1, -1 and so on.
 Three Levels of Correctness

Absolute correctness
 can be demonstrated only by a test that involves every possible
combination of inputs.

 Requires huge amount of time; it is therefore not practical

{However for some programswe can prove correctness mathematically}.

 Imagine the sameabove test for a 32 bit machine(how many combinations

are possible? (Hundreds of Millions).
• You are to develop a small program that reads three integer values from an input dialog. The
three values represent the lengths of the sides of a triangle.
• The program displays a message that states whether the triangle is scalene, isosceles, or
equilateral. Write a set of test cases—specific sets of data—to properly test the program
◦ The set of test data must be handled by the program correctly to be considered a successful program.

Remember that
◦ a scalene triangle is one where no two sides are equal,
◦ whereas an isosceles triangle has two equal sides, and
◦ an equilateral triangle has three sides of equal length.

25
19
int main()
{
float a,b,c; WHAT TEST CASES DO YOU THINK
cin>>a >> b >>c; SHOULD BE THERE?
if(a<(b+c) && b < (a+c) && c< (a+b))
For the input values to represent a triangle, they
{ must be integers greater than 0 where the sum of
cout<<"\nIt is a Triangle.”; any two is greater than the third.
if (a==b && b==c)
cout<<"\nIt is a Equilateral Triangle.”;
if(a==b || a==c || b==c)
cout<<"\nIt is a Isosceles Triangle.”;
else
cout<<"\nIt is a Scalene Triangle.”;
}
else
cout<<"This Triangle is not possible.”;
return 0;
}
20
 Questions to Answer ??
1. Do you have a test case that represents a valid scalene triangle
2. Do you have a test case that represents a valid equilateral triangle?
3. Do you have a test case that represents a valid isosceles triangle?
4. Do you have at least three test cases that represent valid isosceles triangles such that you have
tried all three permutations of two equal sides (such as, 3, 3, 4; 3, 4, 3; and 4, 3, 3)?
5. Do you have a test case in which one side has a zero value?
6. Do you have a test case in which one side has a negative value?
7. Do you have a test case with three integers greater than zero such that the sum of two of the
numbers is equal to the third? (That is, if the program said that 1, 2, 3 represents a scalene triangle, it
would contain a bug.)

21
 Questions to Answer ?? (2)
8 Do you have at least three test cases in category 7 such that you have tried all three permutations where
the length of one side is equal to the sum of the lengths of the other two sides (for example, 1, 2, 3; 1, 3, 2;
and 3, 1, 2)?
9 Do you have a test case with three integers greater than zero such that the sum of two of the numbers is
less than the third (such as 1, 2, 4 or 12,15,30)?
10 Do you have at least three test cases in category 9 such that you have tried all three permutations (for
example, 1, 2,4; 1, 4, 2; and 4, 1, 2)?
11 Do you have a test case in which all sides are zero (0, 0, 0)?
12 Do you have at least one test case specifying noninteger values (such as 2.5, 3.5, 5.5)?
13 Do you have at least one test case specifying the wrong number of values (two rather than three integers,
for example)?
14 For each test case did you specify the expected output from the program in addition to the input values?

22
A very simple program
To do exhaustive testing
◦ Test with all possible values of a, b, & c (the three sides)
◦ And their combinations
◦ 32-bit integer, 232 possible values for one integer
◦ Assuming only integers from 1 to 10, there are 1012 possible values for a triangle.
◦ Testing 1000 cases per second, you would need 317 years!
◦ Test with all the invalid inputs
◦ e.g., All strings ;)

23
A 32-bit integer can represent 232 different values. In decimal terms, that's:
• If testing a trivial program is so complex, what about testing large programs of thousands of
lines of code, e.g., air traffic control software
• Solution?

25
• If testing a trivial program is so complex, what about testing large programs of
thousands of lines of code, e.g., air traffic control software

Solution?
◦ Use of sophisticated testing techniques
◦ Select the test cases that have most chances of triggering a failure
◦ e.g, boundary values

26
 "Bugs lurk in corners
Test case design and congregate at
boundaries ..."

OBJECTIVE ---- to uncover errors Boris Beizer

CRITERIA ----- In a complete manner
CONSTRAINT ------ with a minimum of effort and time

 A successful test -- -- that uncovers an as-yet undiscovered error.

 Minimum number of required tests with 100% functional coverage and 0% redundancy.
 Rich variety of test case design methods
- Cause-effect graphing, Equivalence Class Partitioning, Boundary Analysis, and vendor specific: client/server, OO
test case design.

27
Possible approaches:

Black Box Testing: Tests the software based on its

external behavior and functionality without considering its
internal structure or code. It focuses on validating against
requirements and user expectations.

White Box Testing: Examines the internal structure,

logic, and code of the software. Testers use their
understanding of the implementation to design test cases
that exercise different paths within the code.

Both approaches are essential for comprehensive testing,

with black box testing ensuring that the software meets
user needs and requirements, while white box testing
verifies the correctness of the internal logic and
implementation.
 • Types of errors regarding functional requirements of software:
-- Incorrect or missing functions
-- Interface errors
-- Error in data structure & external data base access
-- Performance errors
-- Initialization & termination errors

Black • No functional requirements NO Black Box Testing.

Box
• Demonstrates that each function is fully operational.
Testing
• Uncovers different kind of errors than white box testing.

• Performed later in the testing process.

36
• Black box techniques derive a set of test cases that satisfy the
following criteria:
1. Test cases reduce the number of additional test cases that
must be designed to achieve reasonable testing
2. Test cases that tell us something about the presence or absence
of classes of errors, rather than errors associated only with
the specific test at hand.

• Black box techniques can supplement the test cases generated

by white box.

37
 Equivalence
class Boundary
partitioning value analysis

How to Design
Test Cases?
Cause / effect
graphing (for
combinations of Error
input Guessing
conditions)

38
 Equivalence Partitioning
• Type of Black Box Testing.

• select the right subset with the highest probability of finding the most errors.

• The input domain of a program is partitioned into a finite number of equivalence

classes such that we can reasonably assume that a test of a representative value
of each class is equivalent to a test of any other value.

• By identifying this as an equivalence class, we are stating that if no error is found

by a test of one element of the set, it is unlikely that an error would be found by a
test of another element of the set.

39
 Example
Travel service offers discounts to travelers based on their age.
◦ 0-4 years 100%
◦ 5-15 years 50%
◦ 16-64 years 0%
◦ 64 years and older 25%
Equivalence classes for age
◦ 0, 1, 2, 3, 4
◦ 5, 6, 7, …15
◦ 16, 17, 18, …64
◦ 65, 66, 67, …120
Similarly for destinations
◦ e.g., destination can be grouped based on regions (that have same fair)
Nothing special for name

40
 Identifying Equivalence classes

 A key concept in the identification of classes is negation, i.e. If a

characteristic is identified as an equivalence class, then one should
immediately negate the characteristic in order to find examples of
classes which should cause the module to do something different such as
“generate an error message”.

41
 Concept of Negation

• Suppose we are developing a software module that calculates the square root of a number. One of
the characteristics we identify is whether the input number is positive. We might initially think that the
class for positive numbers is what we need to focus on.

• So, the equivalence class could be:

• Positive numbers: Numbers greater than zero.

• Now, applying the concept of negation, we consider what doesn't belong to this class. In this case, it
would be non-positive numbers, which include zero and negative numbers.

• So, the negation of the equivalence class would be:

• Non-positive numbers: Numbers less than or equal to zero.
 Identifying Equivalence classes

 Partitioning each input condition into two or more groups.

 Two types of equivalence classes are identified:

1. Valid Equivalence Classes
2. Invalid Equivalence Classes

41
Identifying the Equivalence Classes
We have valid ECs and invalid Ecs

• Design tests such that each valid EC and each invalid EC are included
in at least one test case
• Test cases are based on the specifications not on code

• Better results are obtained in testing boundary values of ECs as well.

• Very important to test boundaries. Rich source of very common errors.

38
Equivalence Partitioning Guidelines
• Range of values condition
– Input spec: itemCount can be from 1 to 999, inclusive.
• identify one valid (1<itemCount<999)
• identify two invalid (itemCount < 1, itemCount>999)

• Countable set of values condition

– Input spec: one through six owners can be listed…
• identify one valid owner
• identify two invalid (zero owners, more the six owners)

39
Identifying the Equivalence Classes

40
Equivalence Class Test Cases

 Consider a numerical input variable, i, whose values may range from -200 through
+200. Then a possible partitioning of testing input variable by 4 people may be:
 -200 to -100
 -101 to 0
 1 to 100
 101 to 200

 Define “same sign” as the equivalence relation, R, defined over the input variable’s
value set, i = {-200 - -,0, - -, +200}. Then one partitioning will be:
 -200 to -1 (negative sign)
 0 (no sign)
 1 to 200 (positive sign)

A sample equivalence test case “set” from the above “same sign” relation would be { -5; 0; 8 }
 Equivalence Partitioning Guidelines

• If the input condition is a range, identify one valid equivalence class member and two
invalid equivalence class members.

• If input condition is a number of values, identify one valid and two invalid.

• If input condition is a set of input values (each to be handled differently), identify one valid
for each and one invalid.

• If input condition specifies a must be situation, identify one valid and one invalid.

• If elements in an equivalence class are not handled in the same manner, split the
equivalence class.

42
 Guidelines for Defining Equivalence Classes

 If an input condition specifies a range, one valid and two invalid equivalence
classes are defined
 Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}
 If an input condition requires a specific value, one valid and two invalid
equivalence classes are defined
 Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}
 If an input condition specifies a member of a set, one valid and one invalid
equivalence class are defined
 Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}
 If an input condition is a Boolean value, one valid and one invalid class are define
 Input: {true condition} Eq classes: {true condition}, {false condition}

43
 Equivalence Class Testing

• Use the mathematical concept of partitioning into equivalence classes to

generate test cases for Functional (Black-box) testing

• The key goals for equivalence class testing are similar to partitioning:
 completeness of test coverage
 lessen duplication of test coverage
 Test Case and the Equivalence Classes

• Formally, one test case per equivalence class should be enough.

• A black box method aimed at increasing the efficiency of testing and,

at the same time, improving coverage of potential error conditions

45
Test Case and the Equivalence Classes
According to the equivalence class partitioning method:

• Each valid EC and each invalid EC are included in at least one test case.

• Note: equivalence CLASS!

 For an acceptable range of input integers is 1-10, we need at least one test with a value in the range
1-10.
 Boundaries: 1 and 10
 Invalid EC classes might be EC with values < 1, another class > 10, another one - negative
numbers, ....

• Note: Run tests for invalid equivalence classes one at a time.

46
 Weak Normal Equivalence Class Testing

• Normal Range: Equivalence classes representing typical or normal input values that the system is
expected to handle correctly. These values fall within the expected range of inputs and are likely to
be processed without errors.

• Weak Range: Equivalence classes representing values that are at the boundaries or just outside
the normal range. These values are chosen to test the behavior of the system at its limits or near
boundaries, where errors or unexpected behavior may occur.

"weak" in this context may stem from the perspective that these values are
not fully representative of the normal or typical range of inputs but rather lie
at the edges where behavior might change or errors might occur. Therefore,
they are termed "weak" to emphasize their significance in testing, as they
help uncover issues related to boundary conditions and edge cases.
 Example

Suppose the system has the following requirements:

1. Packages weighing between 0 and 10 kilograms are charged a standard shipping rate.
2. Packages weighing more than 10 kilograms are charged an additional fee for each kilogram over 10.

Based on these requirements, we can identify the following equivalence classes:

1. Normal Equivalence Classes:

• Packages weighing between 0 and 10 kilograms (inclusive).

2. Weak Equivalence Classes:

1. Packages weighing just below the normal range boundary (e.g., 0 kilograms).
2. Packages weighing just above the normal range boundary (e.g., 10 kilograms).
3. Packages weighing significantly above the normal range (e.g., 11 kilograms).
Example of Weak Equivalence Testing Class
(WETC) *
• Number of WETCs need= Max number of equivalence classes among {A, B, C}
• Given: |A| = 3, |B| = 4, |C| = 2

• 4 WETCs are enough

#test cases = #classes in the partition with the largest numbering of subsets.

48
 Strong Equivalence Classes

• It focuses on testing inputs that are fully representative of the expected behavior, ensuring that
all required components are present and fulfilling the system's requirements.

• The Cartesian product guarantees that we have a notion of “completeness” in following two
ways:

a) We cover all equivalence classes.

b) We have one of each possible combination of inputs.
 Example

Suppose the system has the following requirements:

1. Passwords must be at least 8 characters long.
2. Passwords must contain at least one uppercase letter, one lowercase letter, one digit, and one special character.
Example of Strong Equivalence Class Testing (SECT)

|A| = 3,
|B| = 4,
|C| = 2

# of test cases
◦ 3x4x2= 24

52
Explanation
 Testing for Robustness

• If error conditions are a high priority, we should extend strong

equivalence class testing to include both valid (E) and invalid
inputs (U)
◦ For example: age < 0 and age > 120

• Again, robustness can be applied with WECT and SECT

54
 NextDate Example
• NextDate is a function with three variables: month, day, year. It returns the
date of the day after the input date.
◦ Limitation: 1812-2020

• Treatment Summary:
◦ if it is not the last day of the month, the next date function will simply increment the day
value.
◦ At the end of a month, the next day is 1 and the month is incremented.
◦ At the end of the year, both the day and the month are reset to 1, and the year
incremented.
◦ Finally, the problem of leap year makes determining the last day of a month interesting.

51
Equivalence Classes
Valid Equivalence Classes
◦ M1 = {1 <= month <= 12} Valid months: January (1) to December (12)
◦ D1 = {1 <= day <= 31} Valid days: 1 to 31 (based on the month, considering leap years and months with fewer days)
◦ Y1 = {1812 <= year <= 2020} Valid years: Up to 2020

52
Poor Equivalence Classes
Valid Equivalence Classes
◦ M1 = {1 <= month <= 12}
◦ D1 = {1 <= day <= 31}
◦ Y1 = {1812 <= year <= 2020}

• Should take into account special cases

• Leap year, February

57
Better classes
Better classes
Valid Equivalence Classes
◦ M1 = {1, 3, 5, 7,8, 10, 12}
◦ M2 = {4, 6, 9, 11}
◦ M3 = {2}
◦ D1 = {1 <= day <= 28}
◦ D2 = {29}
◦ D3 = {30}
◦ D4 = {31}
◦ Y1 = {year = 1900}
◦ Y2 = {1812 <= year <= 2020 AND (year != 1900) AND (year mod 4 = 0)}
◦ Y3 = {1812 <= year <= 2020 AND year mod 4 != 0 }

59
Weak Equivalent Class - Test cases
#test cases=maximum partition size (D)=4

55
Explanation
Strong Equivalent Class Testing - Test cases

56