Controlflow Testing
Controlflow Testing
[Reading assignment: Chapter 7, pp. 105-122 plus many things in slides that are not in the book ]
Control-Flow Testing
Control-flow testing is a structural testing strategy that uses the programs control flow as a model. Control-flow testing techniques are based on judiciously selecting a set of test paths through the program. The set of paths chosen is used to achieve a certain measure of testing thoroughness.
E.g., pick enough paths to assure that every source statement is executed as least once.
Motivation
Control-flow testing is most applicable to new software for unit testing. Control-flow testing assumptions:
specifications are correct data is defined and accessed properly there are no bugs other than those that affect control flow
Control Flowgraphs
The control flowgraph is a graphical representation of a programs control structure.
A process block is a sequence of program statements uninterrupted by either decisions or junctions. (i.e., straight-line code).
A process has one entry and one exit. A program does not jump into or out of a process.
Exponentiation Algorithm
1 2 3 4 scanf(%d %d,&x, &y); if (y < 0) pow = -y; else pow = y; z = 1.0; while (pow != 0) { z = z * x; pow = pow - 1; } if (y < 0) z = 1.0 / z; printf (%f,z);
5 6 7
Paths
A path through a program is a sequence of statements that starts at an entry, junction, or decision and ends at another (possible the same), junction, decision, or exit. A path may go through several junctions, processes, or decisions, one or more times. Paths consist of segments. The smallest segment is a link. A link is a single process that lies between 2 nodes.
Paths (Contd)
The length of a path is the number of links in a path. An entry/exit path or a complete path is a path that starts at a routines entry and ends at the same routines exit.
Paths (Contd)
Complete paths are useful for testing because:
It is difficult to set up and execute paths that start at an arbitrary statement. It is difficult to stop at an arbitrary statement without changing the code being tested. We think of routines as input/output paths.
Buggy Code
2.Statement Coverage: For x < 0 the program produces the correct result AND every statement has been executed. 3.Branch Coverage: Would have found the bug! Therefore 2 does not imply 3.
Branch Coverage: Does not exercise dead code. Therefore 3 does not imply 2. However, 3 implies 2 for programs written in a structured programming language without goto statements.
P1 ):
3) Branch Testing ( P2 ):
100% branch coverage. Execute enough tests to assure that every branch alternative has been exercised at least once under some test.
P P ... P 1 2
Quote
The more we learn about testing, the more we realize that statement and branch coverage are minimum floors below which we dare not fall, rather that ceilings to which we should aspire. - B. Beizer.
You must pick enough paths to achieve statement and branch coverage. Question: What is the fewest number of paths to achieve statement and branch coverage? Answer: Unask the question.
It is better to take many simple paths than a few complicated ones. There is no harm in taking paths that will exercise the same code more than once.
Which Paths?
3
i
4
h l T
5
g
6
f
10
k F m
j T
7
F
Guidelines
Select paths as small variations of previous paths. Try to change one thing in each path at a time.
Control-flow testing is more effective for unstructured code than for code that follows structured programming. Experienced programmers can bypass drawing flowgraphs by doing path selection on the source.
Test Outcomes
The outcome of test is what we expect to happen as a result of the test. Test outcomes include anything we can observe in the computers memory that should have (not) changed as a result of the test. Since we are not kiddie testing we must predict the outcome of the test as part of the test design process.
Testing Process
run the test observe the actual outcome compare the actual outcome to the expected outcome.
Path Instrumentation
All instrumentation methods are a variation on a theme of an interpretive trace. An interpretive trace program executes every statement in order and records:
the intermediate values of all calculations the statement labels traversed ...
a abc adc
INPUT
OUTPUT -x x
F 3
b T
INPUT
abc adc T F A Negative Integer, x A Positive Integer, x
d a F 3 b T c
OUTPUT
-x x
/* COUNT This program counts the number of characters and lines in a text file. INPUT: Text File OUTPUT: Number of characters and number of lines. */ 1 main(int argc, char *argv[]) 2 { 3 int numChars = 0; 4 int numLines = 0; 5 char chr; 6 FILE *fp = NULL; 7
12
13 14 15 16 17 18
}
fp = fopen(argv[1], r); if (fp == NULL) { perror(argv[1]); return (-2); }
/* display error message */
20
21 22 23 24 25 26
{
chr = getc(fp); if (chr == \n) ++numLines; else ++numChars; } /* read character */ /* if carriage return */
The junction at line 12 and line 18 are not needed because if you are at these lines then you must also be at line 14 and 19 respectively.
ab
None
agc
Invalid Input Filename Input File with one character and no Carriage Return at the end of the line Input file with no characters and one carriage return
aghdj kli
aghd efli
ab
None
agc
Invalid Input Filename T, F F Input File with one character and no Carriage Return at the end of the line Input file with no characters and one carriage return
aghdjkli
aghdefli
T, F
Summary
The object of control-flow testing is to execute enough tests to assure that statement and branch coverage has been achieved. Select paths as deviation from the normal paths. Add paths as needed to achieve coverage. Use instrumentation (manual or using tools) to verify paths. Document all tests and expected test results. A test that reveals a bug has succeeded, not failed.