BITS Pilani

K K Birla Goa Campus

CSF111: Computer Programming

Testing

Swaroop Joshi

2023
Testing a function
/**
* @brief Computes the area of a rectangle
given its length and width.
* Requires: length > 0, width > 0
* @return Product of length and width

*
* @retval 4 area_rectangle(2, 2)
* @retval 24 area_rectangle(4, 6)
* @retval 168 area_rectangle(12, 14)
*
*/
int area_rectangle(int length, int width) {
return length * width;
}
Testing a function
/**
* @brief Computes the area of a rectangle
(a) Designing good test cases
given its length and width.
* Requires: length > 0, width > 0 (b) Running your function against
* @return Product of length and width those test cases

*
* @retval 4 area_rectangle(2, 2)
* @retval 24 area_rectangle(4, 6)
* @retval 168 area_rectangle(12, 14)
*
*/
int area_rectangle(int length, int width) {
return length * width;
}
Designing good test cases
Designing good test cases

✤ What does it mean for a program unit (say, a function) to be correct?

Designing good test cases

✤ What does it mean for a program unit (say, a function) to be correct?

✤ It does what it is supposed to do.

✤ It doesn’t do what it is not supposed to do.

✤ “Supposed to do” ?

✤ Purpose/contract = intended behaviour

Example function

/**
* @brief Reports some factor of the given number.
*
* Requires: n > 0
*/
int a_factor(int n) {

}
Example function

/**
* @brief Reports some factor of the given number.
*
* Requires: n > 0
*/
int a_factor(int n) {

return 1;

}
Example function

/**
* @brief Reports some factor of the given number.
*
* Requires: n > 0
*/
int a_factor(int n) {

return 1; Is this function body correct?

}
Behaviours

Allowed
Actual behaviours of
behaviours of the
the function
function
(body)
(purpose/contract)
Behaviours

Contract of a_factor allows:

n = 12, a_factor = 4
(12, 4)
(Name of the function
Allowed
Actual behaviours of
represents the return value) of the
behaviours
the function
function
(body)
(purpose/contract)
Behaviours
Contract of a_factor forbids:
n = 12, a_factor = 5

(12, 5)
(12, 4)
Allowed
Actual behaviours of
behaviours of the
the function
function
(body)
(purpose/contract)
Behaviours

(12, 5)
(12, 4)
Allowed
Actual behaviours of
behaviours of the
the function
function
(body)
Contract of a_factor(purpose/contract)
allows:
n = 12, a_factor = 6
(12, 6)
Behaviours

(12, 5)
(12, 4)
Allowed
Actual behaviours of
behaviours of the
the function
function
(12, 1)
(body)
(purpose/contract)

(12, 6) Contract of a_factor allows:

n = 12, a_factor = 1
Behaviours

(12, 5)
(12, 4)
Allowed
Actual behaviours of
behaviours of the
the function
function Body of a_factor
(12, 1) gives:
(body)
(purpose/contract)

(12, 6)
Correctness = if actual is a subset of allowed

Allowed
Actual behaviours of
behaviours of the
the function
function
(body)
(purpose/contract)
Testing
Testing

✤ Testing is a technique for trying to refute the claim that a function body is
correct for the function contract
Testing

✤ Testing is a technique for trying to refute the claim that a function body is
correct for the function contract

✤ In other words, the goal of testing is to show that the function body does
not correctly implement the contract, i.e., that it is defective

✤ As a tester, you really want to think this way!

Psychology of Testing
Psychology of Testing

✤ Design and coding are creative activities

Psychology of Testing

✤ Design and coding are creative activities

✤ Testing is a destructive activity

✤ The primary goal is to “break” the software, i.e., to show that it has defects
Psychology of Testing

✤ Design and coding are creative activities

✤ Testing is a destructive activity

✤ The primary goal is to “break” the software, i.e., to show that it has defects

✤ Very often the same person does both coding and testing (not a best practice)

✤ You need a “split personality”: when you start testing, become paranoid and
malicious

✤ It’s surprisingly hard to do: people don’t like nding out that they made mistakes
fi
Psychology of Testing “Program testing can be used to show
the presence of bugs, but never to show
their absence!”
✤ Design and coding are creative activities — Edsger W. Dijkstra (1972)

✤ Testing is a destructive activity

✤ The primary goal is to “break” the software, i.e., to show that it has defects

✤ Very often the same person does both coding and testing (not a best practice)

✤ You need a “split personality”: when you start testing, become paranoid and
malicious

✤ It’s surprisingly hard to do: people don’t like nding out that they made mistakes
fi
Incorrect code = actual NOT a subset of allowed

Allowed
behaviours of the
Actual behaviours of
function
the function
(purpose/contract)
(body)
Test cases

✤ Each input value and the corresponding allowed/expected result is a test case

✤ To make testing most likely to succeed in revealing defects, best practices include:

✤ Test boundary cases: “smallest”, “largest”, “special” values based on the

contract

✤ Test routine cases

✤ Test challenging cases, i.e., ones that, if you were writing the code (maybe you
didn’t write the code being tested!), you might nd dif cult or error-prone
fi
fi
Example test cases for a_factor

n Expected return value Reason

1 1 Boundary

Routine
2 1 or 2
Or challenging (prime)
Routine
4 1, 2, or 4
Or challenging (square)

12 1, 2, 3, 4, 6, or 12 Routine

…
Testing a function
/**
* @brief Computes the area of a rectangle
(a) Designing good test cases
given its length and width.
* Requires: length > 0, width > 0 (b) Running your function against
* @return Product of length and width those test cases

*
* @retval 4 area_rectangle(2, 2)
* @retval 24 area_rectangle(4, 6)
* @retval 168 area_rectangle(12, 14)
*
*/
int area_rectangle(int length, int width) {
return length * width;
}
Running your function against test-cases
#include<stdio.h>

/** ... */
int area_rectangle(int length, int width) {
return length * width;
}

int main() {
int length, width;
printf("Enter the length and width of a rectangle: ");
scanf("%d%d", &length, &width);
printf("Area of a rectangle %dx%d is %d\n",
length, width, area_rectangle(length, width));
return 0;
}
Running your function against test-cases
#include<stdio.h>

/** ... */
int area_rectangle(int length, int width) {
return length * width; Testing with printfs
}
Simple but tedious
int main() {
int length, width;
printf("Enter the length and width of a rectangle: ");
scanf("%d%d", &length, &width);
printf("Area of a rectangle %dx%d is %d\n",
length, width, area_rectangle(length, width));
return 0;
}
Running your function against test-cases
#include<stdio.h>

/** ... */
int area_rectangle(int length, int width) {
return length * width;
}

int main() {
if (4 == area_rectangle(2, 2)) {
printf("[PASS] Area of rectangle 2x2 is 4\n");
} else {
printf("[FAIL] Area of rectangle 2x2 is 4\n");
}

return 0;
}
Running your function against test-cases
#include<stdio.h>

/** ... */
int area_rectangle(int length, int width) {
return length * width; Testing with if-else
}
Let the computer do the work!
int main() {
if (4 == area_rectangle(2, 2)) {
printf("[PASS] Area of rectangle 2x2 is 4\n");
} else {
printf("[FAIL] Area of rectangle 2x2 is 4\n");
}

return 0;
}
Running your function against test-cases
#include<stdio.h>

/** ... */
int area_rectangle(int length, int width) {
return length * width; Testing with if-else
}
Let the computer do the work!
int main() {
if (4 == area_rectangle(2, 2)) {
printf("[PASS] Area of rectangle 2x2 is 4\n");
} else {
printf("[FAIL] Area of rectangle 2x2 is 4\n");
}

return 0; N test cases = N if-else blocks

}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h> Testing using functions

void check(bool expr, char* message) { Let the computer do even more work!
if (expr) { printf("[PASS] "); } But be careful when writing the tests
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h> void: a data type that signi es nothing
#include<stdbool.h>
Use as a return type of functions that don’t
void check(bool expr, char* message) {
evaluate to something meaningful
if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
fi
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message); bool: a type for true or false (C99)
} Must #include<stdbool.h>

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h> char*: character string
#include<stdbool.h> Details later

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message); %s: placeholder for strings
}

int area_rectangle(int length, int width) { ... }

int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");

return 0;
}
Running your function against test-cases
#include<stdio.h>
#include<stdbool.h>

void check(bool expr, char* message) {

if (expr) { printf("[PASS] "); }
else { printf("[FAIL] "); }
printf("%s", message);
}

int area_rectangle(int length, int width)N{test

... } = N calls to check
cases
int main() {
check(4 == area_rectangle(2, 2), "Area of rectangle 2x2 is 4");
check(24 == area_rectangle(4, 6), "Area of rectangle 4x6 is 24");
return 0;
}
Comparing real numbers

✤ Problems with precision

✤ We have in nite real numbers in math: even in the range 0.1 and 0.2

✤ Cannot possible represent all of them in a computer

✤ What we get is an approximate representation of numbers

✤ Better to use relative errors when comparing real numbers

fi
Factorial using Gosper’s formula

/**
* @brief Computes the approximate
* factorial of the given number using
* Gosper's formula.
* Requires: n>=0
*/
double factorial(int n) {
double term1 = pow(n, n);
double term2 = pow(M_E, -n);
double subterm = 2 * n + (double) 1/3;
double term3 = sqrt(subterm * M_PI);
return term1 * term2 * term3;
}
Factorial using Gosper’s formula

/** M_E and M_PI are constants de ned in

* @brief Computes the approximate math.h
* factorial of the given number using
* Gosper's formula.
* Requires: n>=0
*/
double factorial(int n) {
double term1 = pow(n, n);
double term2 = pow(M_E, -n);
double subterm = 2 * n + (double) 1/3;
double term3 = sqrt(subterm * M_PI);
return term1 * term2 * term3;
}
fi
Factorial using Gosper’s formula

/**
* @brief Computes the approximate
* factorial of the given number using Type-casting: changing the type of an
* Gosper's formula.
expression to another type
* Requires: n>=0
*/
double factorial(int n) { Here, int → double
double term1 = pow(n, n);
double term2 = pow(M_E, -n); 1/3: / is an int operator, gives the
double subterm = 2 * n + (double) 1/3; quotient
double term3 = sqrt(subterm * M_PI); (double) 1 makes it a real-number
return term1 * term2 * term3;
operation, gives the expected result
}
Factorial using Gosper’s formula
Computes 5! to 119.97003
Acceptable value?

/**
Cannot use check(120==factorial(5), …)
* @brief Computes the approximate
* factorial of the given number using
* Gosper's formula.
* Requires: n>=0
*/
double factorial(int n) {
double term1 = pow(n, n);
double term2 = pow(M_E, -n);
double subterm = 2 * n + (double) 1/3;
double term3 = sqrt(subterm * M_PI);
return term1 * term2 * term3;
}
Factorial using Gosper’s formula

/**
* @brief Reports if the given numbers are within 1%
* relative error of each other.
*/
bool approx_eq(double expected, double actual) {
const double EPS = .01;
double relative_error = fabs(expected - actual)/actual;
return relative_error < EPS;
}
Factorial using Gosper’s formula

/**
* @brief Reports if the given numbers are within 1%
* relative error of each other.
*/
bool approx_eq(double expected, double actual) {
const double EPS = .01;
double relative_error = fabs(expected - actual)/actual;
return relative_error < EPS;
}

int main() {
check(approx_eq(120, factorial(5)), "5! = 120\n");
return 0;
}
Factorial using Gosper’s formula

/**
* @brief Reports if the given numbers are within 1%
* relative error of each other.
*/
bool approx_eq(double expected, double actual) {
const double EPS = .01;
double relative_error = fabs(expected - actual)/actual;
return relative_error < EPS;
}
fabs returns the absolute value of the
given real number
int main() {
check(approx_eq(120, factorial(5)), "5! = 120\n");
return 0;
}
Factorial using Gosper’s formula
Call approx_eq on the expected and the
actual values and pass the returned value
to the check function
/**
* @brief Reports if the given numbers are within 1%
* relative error of each other.
*/
bool approx_eq(double expected, double actual) {
const double EPS = .01;
double relative_error = fabs(expected - actual)/actual;
return relative_error < EPS;
}

int main() {
check(approx_eq(120, factorial(5)), "5! = 120\n");
return 0;
}
Testing summary
Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

✤ Testing can show the presence of bugs but cannot prove their absence
Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

✤ Testing can show the presence of bugs but cannot prove their absence

✤ We discussed three methods of running the tests

Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

✤ Testing can show the presence of bugs but cannot prove their absence

✤ We discussed three methods of running the tests

✤ Testing using printfs; Testing using if-else; Testing using a helper function
Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

✤ Testing can show the presence of bugs but cannot prove their absence

✤ We discussed three methods of running the tests

✤ Testing using printfs; Testing using if-else; Testing using a helper function

✤ Use the simpler ones if you are more comfortable with them
Testing summary

✤ Correctness = actual behaviour is a subset of the allowed behaviour

✤ Testing is a counterintuitive task

✤ Testing can show the presence of bugs but cannot prove their absence

✤ We discussed three methods of running the tests

✤ Testing using printfs; Testing using if-else; Testing using a helper function

✤ Use the simpler ones if you are more comfortable with them

✤ More imp: doing systematic testing in some form