1 2

Content
1. Testing overview
ITSS SOFTWARE DEVELOPMENT 2. Unit Test
08. UNIT TEST
3. Integration Test

1 2

3 4

Software Quality Assurance (QA)

Testing Testing plus other activities including
 “[T]he means by which the presence, quality, or • Static analysis (assessing code without executing it)
genuineness of anything is determined; a means of trial.” – • Proofs of correctness (theorems about program properties)
dictionary.com • Code reviews (people reviewing others’ code)
 A software test executes a program to determine whether a • Software process (placing structure on the development
property of the program holds or doesn’t hold lifecycle)
 A test passes [fails] if the property holds [doesn’t hold] on • …and many more ways to find problems and to increase
that run confidence

No single activity or approach

can guarantee software quality

3 4

1
 5 6

V Model – Different test level Test levels

• Unit test: ONE module at a time • Unit Testing: Does each unit (class, method, etc.) do
• Integration test: The linking modules what it supposed to do?
• System test: The whole (entire) system • Smallest programming units
• Strategies: Black box and white box testing
• Techniques, Tools
• Integration Testing: do you get the expected results
when the parts are put together?
• Strategies: Bottom-up, top-down testing
• System Testing: does it work within the overall
system?
• Acceptance Testing: does it match to user needs?

5 6

7 8

2.1. Unit test approaches

Content Black box and White box testing
A. Choose input data (“test inputs”)
1. Testing overview B. Define the expected outcome (“soict”)
2. Unit Test C. Run the unit (“SUT” or “software under test”) on the input
and record the results
3. Integration Test D. Examine results against the expected outcome (“soict”)

Specification
Precondition Postcondition
Implementation = soict?

Black box White box

Must choose inputs without Can choose inputs with
knowledge of the knowledge of the
implementation implementation

7 8

2
 9 10

It’s not black-and-white, but… Terms

White box
Black box
Can choose inputs with
Must choose inputs without
knowledge of the • Test case
knowledge of the implementation
implementation • a set of conditions/variables to determine whether a
system under test satisfies requirements or works
• Has to focus on the • Common use: coverage
correctly
behavior of the SUT • Basic idea: if your test
• Test suite
• Needs an “soict” suite never causes a
• a collection of test cases related to the same test
• Or at least an statement to be work
expectation of executed, then that
• Test plan
whether or not an statement might be
• a document which describes testing approach and
exception is thrown buggy
methodologies being used for testing the project,
risks, scope of testing, specific tools

9 10

11 12

Test suite Unit Testing techniques

• Example of test suite • For test case design
• Test case 1: Login • (2.2) Test Techniques for Black Box Test
• Test case 2: Add New Products • Equivalence Partitioning Analysis
• Boundary-value Analysis
• Test case 3: Checkout
• Decision Table
• Test case 4: Logout
• Use Case-based Test
• (2.3) Test Techniques for White Box Test
• Control Flow Test with C0, C1 coverage
• Sequence chart coverage test

11 12

3
 14

2.2. Blackbox Testing Techniques

Example: Examination Judgment Program
2.2.1. Equivalence Partitioning
• Create the encompassing test cases by analyzing the input • Program Title: “Examination Judgment Program”
data space and dividing into equivalence classes • Subject: Two subjects as Mathematics, and Physics
• Input condition space is partitioned into equivalence classes Judgment
• Every input taken from a equivalence class produces the same
• Specification:
result
• Passed if
• scores of both mathematics and physics are greater than or equal to 70
out of 100
or,
• average of mathematics and physics is greater than or equal to 80 out
of 100
• Failed => Otherwise

13 14

15 16

Equivalence Partitioning of Input space and test cases

Equivalence Partitioning
Discussions and additional analysis
• 7 equivalence classes => at least 7 test cases/data • Are we successful?
• No we don’t! Why?
Score of Average Score
Physics. is 80. • →One thing is missing!
100
(2)
(5)
• The scope of input space analyzed is not enough!
(1)
80
Score Math. Physics Result • We must add “Invalid value” as the test data.
(4)
70
(1) 55 85 Failed • For example, some patterns of “Invalid value”.
(3)
(2) 67 97 Passed • (8) Math = -15, Physics = 120 Both score are invalid.
60
Average
Score (3) 96 68 Passed • (9) Math = 68, Physics = -66 Physics score is invalid.
is 80.
(4) 77 80 Passed • (10) Math = 118, Physics = 85 Math score is invalid.
40
(7) (6)
(5) 85 92 Passed
(6) 79 58 Failed
20
(7) 52 58 Failed
Score of
Math.

0 20 40 60 70 80 100

15 16

4
 17

More equivalent classes Analysis and discussions

• Additional 3 test cases/data • We tried to create encompassing test cases based on
(8) Average Score Some invalid data are added.
external specification.
• Successful? “Yes” !
is 80.
100
(2)
Score of
Physics.
(1)
(5) (10) Score Math. Physics Result
80 (1) 55 85 Failed • Next question. The test cases/data are fully effective?
(4)
70 (2) 67 97 Passed
(3) • We have to focus on the place in which many defects are there,
(3) 96 68 Passed
60
Average don’t we?
Score (4) 77 80 Passed
is 80. • Where is the place ?
40
(5) 85 92 Passed
(7) (6)
(6) 79 58 Failed
20
(7) 52 58 Failed → “Boundary-value analysis”
(8) -15 120 Invalid
Score of
Math. (9) 68 -66 Invalid
0 20 40 60 70 80 100
(10) 118 85 Invalid
(9)

17 18

19 20

2.2. Blackbox Testing Techniques

2.2.2. Boundary-value analysis
Example: Boundary-value analysis
• Extract test data to be expected by analyzing boundary • Boundary values of the mathematics score of small
input values => Effective test data case study:
• Boundary values can detect many defects effectively
→E.g. mathematics/physics score is 69 and 70 Invalid value failed passed Invalid value
• The programmer has described the following wrong code: -1 0 69 70 100 101
if (mathscore > 70) {
................... • What about the boundary value analysis for the
} average of mathematics and physics?
• Instead of the following correct code;
if (mathscore >= 70) { failed passed
...................
79 80
}

19 20

5
 2.2. Blackbox Testing Techniques
2.2.3. Decision Table
Example: Decision table
• Relations between the conditions for and the contents of • The conditions for creating reports from employee files
the processing are expressed in the form of a table
• A decision table is a tabular form tool used when complex
conditions are combined Under age 30 Y Y N N
Male Y N Y N
Married N Y Y N
Output Report 1 - X - -
Output Report 2 - - - X
Output Report 3 X - - -
Output Report 4 - - X -

21 22

Decision Table for “Examination Judgement”??? Decision Table for “Examination Judgement”

Test Data from Equivalence Analysis Condition1: Mathematics score=>70

(8) Average Score
is 80. Condition2: Physics score=>70
100
(2) Condition3: Average of Mathematics, and Physics =>80
Score of Score Math. Physics Result
Physics. (5) (10) ----------------------TC5---------TC4---------TC3---------TC6----------TC2--------TC1----------TCNG--------TC7
(1) (1) 55 85 Failed Condition1 True True True True False False False False
80
(4) (2) 67 97 Passed Condition2 True True False False True True False False
70 Condition3 True False True False True False False
(3) (3) 96 68 Passed True(none)

60 ---------------------------------------------------------------------------------------------------------------------------------------
Average (4) 77 80 Passed
Score “Passed” Yes Yes Yes --- Yes --- N/A --
is 80. (5) 85 92 Passed “Failed” --- --- --- Yes --- Yes N/A Yes
40
(7) (6) (6) 79 58 Failed
(7) 52 58 Failed
20 (8) 15 120 Invalid
Score of (9) 68 -66 Invalid
Math.
(10) 118 85 Invalid
0 20 40 60 70 80 100
(9)

23 24

6
 26

2.2. Blackbox Testing Techniques

Decision Table for “Examination Judgement”
2.2.4. Testing for Use case
• Invalid input data (integer) • ???
• Condition1: Mathematics score = valid that means “0=< the score =< 100”
• E.g. Decision table for Login
• Condition2: Physics score = valid that means “0=< the score =< 100”
• Conditions
--------------------------------TCI1----------TCI2--------TCI3----------TCI4--------
• ???
Condition1 Valid Invalid Valid Invalid
• Results
Condition2 Valid Valid Invalid Invalid -------
• ???
---------------------------------------------------------------------------------
“Normal results” Yes --- --- --- • E.g. Boundary Value Analysis
“Error message math” --- Yes --- Yes • ?
“Error message phys” --- --- Yes Yes

If both of mathematics score and physics score are invalid, two messages are
expected to be output. Is it correct specifications? Please confirm it?

25 26

Test cases for “Log in” Creating test cases from use cases
• “Thành công” • Identify all of the scenarios for the given use case
• Mã PIN đúng
• Alternative scenarios should be drawn in a graph fo
• “Thất bại”
each action
• Mã PIN sai và số lần sai < 3
• “Khoá tài khoản” • Create scenarios for
• Mã PIN sai và số lần sai = 3 • a basic flow,
• one scenario covering each alternative flow,
Mã PIN đúng Y Y N N
• and some reasonable combinations of alternative
Số lần sai < 3 Y N Y N
“Thành công” x N/A - -
flows
“Thất bại” - N/A x - • Create infinite loops
“Khoá tài khoản” - N/A - x
• Phân tích vùng biên? Số lần sai = 2, 4 (?)

27 28

7
 E.g. Control flow test of “Examination Judgment
2.3. White Box Testing Techniques Program”
• Test cases should cover all processing structure in Start

code Get a score

=> Typical test coverage

False
Mathematics
• C0 measure: Executed statements #/all statements #
=>70
• C0 measure at 100% means “all statements are executed”
True
• C1 measure: Branches passed #/all blanches # Physics False M+P/2
False
• C1 measure at 100% means “all branches are executed” =>70 =>80
True True

=> Prevent statements/blanches from being left as non-

“Passed” “Failed”
tested parts
=> Cannot detect functions which aren't implemented
End

29 30

E.g. Control flow test of “Examination Judgment E.g. Control flow test of “Examination Judgment
Program” – 100% C0 coverage Program” – 100% C1 coverage
TC1int: Math score: 88, Physics score: 75 → “Passed” TC1int: Math score: 88, Physics score: 75 → “Passed”
Start Start TC2int: Math score: 68, Physics score: 90 → “Failed”
TC2int TC2int: Math score: 68, Physics score: 90 → “Failed”
TC1int TC1int TC2int TC3int: Math score: 98, Physics score: 68 → “Passed”
Get a score Get a score
All statements are executed. TC3int All branches are executed.
False ALL False ONLY 3 TEST
Mathematics Mathematics
=>70 BRANCHES =>70 CASES???
? Compared to 7 TCs?
True True
False False False
Physics False M+P/2 Physics M+P/2
=>70 =>80 =>70 =>80
True True True True

“Passed” “Failed” “Passed” “Failed”

End End

31 32

8
 E.g. Control flow test of “Examination Judgment
Decision Table for “Examination Judgement”
Program” – 100% C1 coverage
Condition1: Mathematics score=>70 TC2 is covered by TC2int and TC3int?
Start
Condition2: Physics score=>70 TC2intTC2 TC6 is covered by TC3int and TC2int?
TC1int
Condition3: Average of Mathematics, and Physics =>80 Get TC6
a score
TC3int BLACK
----------------------TC5---------TC4---------TC3---------TC6----------TC2--------TC1----------TCNG--------TC7
Condition1 True True True True False False False False
False
Condition2 True True False False True True False False
Mathematics BOX!!!
=>70
Condition3 True False True False True False True(none) False
--------------------------------------------------------------------------------------------------------------------------------------- True
“Passed” Yes Yes Yes --- Yes --- N/A -- False False
“Failed” --- --- --- Yes --- Yes N/A Yes Physics M+P/2
=>70 =>80
• One TCxint can cover plural TCs, based on the correct control flow structure True True
• TC1int covers TC5 and TC4
• TC2int covers TC1 and TC7
• TC3int covers to TC3. “Passed” “Failed”
• TC2 and TC6 are left in no execution.

End

33 34

E.g. Control flow test of “Examination Judgment

Program” – 100% C1 coverage
Data/message path test for integrated test
Mistake ??? => TC1int, TC2int and TC3int enough? • Execute white box test using sequence chart for integration
Start
TC1int TC2int Only TC5 can’t detect them test.
Get a score
=> Both TC4,TC5 are needed Execute every message path/flow
TC3int
False 100% message path/flow coverage
Mathematics
=>70
True • Can apply to other data/message path/flow charts or
False False diagrams
Physics M+P/2
=>70 =>80
True True

“Passed” “Failed”

End

35 36

9
 38

How to test a loop structure program Examples for “Examination Judgment Program”

• For the control flow testing in the software including a  Input two subjects scores, Mathematics and Physics,
loop, the following criteria are usually adopted instead for each member of one class.
of C0/C1 coverage measures.  The input form is “tabular form”.
• Skip the loop.  Class members can be allowed only 0 (zero) through 50.
• Only one pass through the loop.  Output/Print out the “Examination result report for a
• Typical times m passes through the loop class”.
• n, n-1, n+1 passes through the loop  The output form is also “tabular form” that has the columns
such as student name, scores (Math., Physics), passed or
• n is maximum number, m is typical number (m<n)
failed.
• Example: 6 cases based on boundary-value analysis:

0 1 typical times: m n-1 n n+1

37 38

Examples for “Examination Judgment Program” Examples for “Examination Judgment Program”
Start
Exercise: Loop test cases of the module are; n = 50.
Get “number of students” Create the test cases using the
criteria two pages before based “number of students” = 0,
counter = zero
on the following assumptions. “number of students” = 1,
1. “Examination “number of students” = 20,
counter < False Judgment
number of students +1 program” are “number of students” = 49,
True already tested. “number of students” = 50,
2. Input data of this
Function: “Examination Print out “number of students” = 51 → Invalid.
module are
Judgment program” “examination result
already checked,
report for a class”.
and valid.
counter = counter 0 1 typical times: m=20 n-1 n n+1
+1

End
39 40

10
 How to test for nested loops structure 2.4. Combination of Black/White Box test
• Advantage of Black box
At first, first loop’s control • Encompassing test based on external specification
number is determined at • Very powerful and fundamental to develop high-quality software
typical number, and second • Advantage of White box
loop is tested as a simple loop. • If any paths/flows don’t appear in the written specifications, the
paths/flows might be missed in the encompassing tests => White
Next, second loop’s control box test
number is determined at • for data of more than two years before => alternative paths

typical number, and first loop • “0 =< score =< 100” => code: “if 0 =< score ” and “if score =< 100”

is tested as a simple loop.

41 42

44

How to carry out efficient and sufficient test 2.5. JUnit

• First, carry out tests based on the external specifications • A tool for test-drivent development (junit.org)
• If all test cases are successful • JUnit test generators now part of many Java IDEs
=> All external specifications are correctly implemented (Eclipse, BlueJ, Jbuilder, DrJava)
• Second, carry out tests based on the internal specifications • XUnit tools have since been developed for many other
• Add test cases to execute the remaining paths/flow,
languages (Perl, C++, Python, Visual Basic, C#, …)
within external specifications
• If all test cases are successful with coverage = 100%
=> All functions specified in the external specification are
successfully implemented without any redundant codes

43 44

11
 Architectural overview
Why create a test suite?
• JUnit test framework is a
• Obviously you have to test your code—right?
package of classes that lets
• You can do ad hoc testing (running whatever tests occur to you at you write tests for each
the moment), or method, then easily run
• You can build a test suite (a thorough set of tests that can be run those tests
at any time)
• TestRunner runs tests and
• Disadvantages of a test suite
reports TestResults
• It’s a lot of extra programming
• You test your class by
• True, but use of a good test framework can help quite a bit
extending abstract class
• You don’t have time to do all that extra work TestCase
• False! Experiments repeatedly show that test suites reduce
debugging time more than the amount spent building the test suite • To write test cases, you
• Advantages of a test suite
need to know and
understand the
• Reduces total number of bugs in delivered code
Assert class
• Makes code much more maintainable and refactorable

45 46

Writing methods in TestCase

Writing a TestCase
• Pattern follows programming by contract paradigm:
• To start using JUnit, create a subclass of TestCase, to
• Set up preconditions
which you add test methods • Exercise functionality being tested
• Here’s a skeletal test class: • Check postconditions
• Example:
import junit.framework.TestCase; public void testEmptyList() {
public class TestBowl extends TestCase { Bowl emptyBowl = new Bowl();
assertEquals(“Size of an empty list should be zero.”,
} //Test my class Bowl 0, emptyList.size());
assertTrue(“An empty bowl should report empty.”,
emptyBowl.isEmpty());
• Name of class is important – should be of the form }
TestMyClass or MyClassTest • Things to notice:
• Specific method signature – public void testWhatever()
• This naming convention lets TestRunner • Allows them to be found and collected automatically by JUnit
automatically find your test classes • Coding follows pattern
• Notice the assert-type calls…

47 48

12
 Assert methods Assert methods
• assertTrue(String message, Boolean test)
• Assert methods dealing with floEach assert • assertFalse(String message, Boolean test)
method has parameters like these: • assertNull(String message, Object object)
message, expected-value, actual-value • assertNotNull(String message, Object object)
• assertEquals(String message, Object expected, Object actual)
• ating point numbers get an additional argument, (uses equals method)
a tolerance • assertSame(String message, Object expected, Object actual)
• Each assert method has an equivalent version (uses == operator)
• assertNotSame(String message, Object expected, Object
that does not take a message – however, this use actual)
is not recommended because:
• messages helps documents the tests
• messages provide additional information when
reading failure logs

49 50

More stuff in test classes JUnit tests for Counter

public class CounterTest extends junit.framework.TestCase {
• Suppose you want to test a class Counter
Counter counter1;
• public class CounterTest public CounterTest() { } // default constructor
extends junit.framework.TestCase {
• This is the unit test for the Counter class protected void setUp() { // creates a (simple) test fixture
• public CounterTest() { } //Default constructor counter1 = new Counter();
}
• protected void setUp()
• Test fixture creates and initializes instance variables, etc. public void testIncrement() { Note that each test begins
• protected void tearDown() assertTrue(counter1.increment() == 1); with a brand new counter
• Releases any system resources used by the test fixture assertTrue(counter1.increment() == 2);
} This means you don’t
• public void testIncrement(), public void testDecrement()
have to worry about the
• These methods contain tests for the Counter methods increment(), public void testDecrement() { order in which the tests
decrement(), etc. assertTrue(counter1.decrement() == -1); are run
• Note capitalization convention }
}

51 52

13
 TestSuites JUnit in Eclipse
• TestSuites collect a selection of tests to run them as a unit
• To create a test class,
• Collections automatically use TestSuites, however to specify the order
in which tests are run, write your own: select File→ New→
public static Test suite() {
suite.addTest(new TestBowl(“testBowl”));
Other... → Java, JUnit,
suite.addTest(new TestBowl(“testAdding”)); TestCase and enter the
return suite;
}
name of the class you
• Should seldom have to write your own TestSuites as each method in will test
your TestCase should be independent of all others
• Can create TestSuites that test a whole package:
public static Test suite() { Fill this in
TestSuite suite = new TestSuite();
suite.addTestSuite(TestBowl.class);
suite.addTestSuite(TestFruit.class);
return suite; This will be filled in
}
automatically

53 54

Running JUnit Results

Your results are here

Second, use this First, select a Test class

pulldown menu

Third, Run As → JUnit Test

55 56

14
 58

Unit testing for other languages

Another example: sqrt
• Unit testing tools differentiate between: // throws: IllegalArgumentException if x < 0
// returns: approximation to square root of x
• Errors (unanticipated problems caught by exceptions) public double sqrt(double x)
• Failures (anticipated problems checked with assertions)
What are some values or ranges of x
• Basic unit of testing:
that might be worth testing
• CPPUNIT_ASSERT(Bool) examines an expression
• CPPUnit has variety of test classes  x < 0 (exception thrown)
(e.g. TestFixture)  x ≥ 0 (returns normally)
• Inherit from them and overload methods  around x = 0 (boundary condition)
 perfect squares (sqrt(x) an integer), non-perfect squares
 x < sqrt(x), x > sqrt(x)
 Specific tests: say x = {-1, 0, 0.5, 1, 4}

57 58

59 60

Subdomains Testing RandomHello

• Many executions reflect the same behavior – for sqrt, for • “Create your first Java class with a main method that
example, the expectation is that will randomly choose, and then print to the console,
• all x < 0 inputs will throw an exception one of five possible greetings that you define.”
• all x ≥ 0 inputs will return normally with a correct answer • We’ll focus on the method getGreeting, which
• By testing any element from each subdomain, the intention randomly returns one of the five greetings
is for the single test to represent the other behaviors of the
• We’ll focus on black-box testing – we will work with no
subdomain – without testing them!
knowledge of the implementation
• Of course, this isn’t so easy – even in the simple example
above, what about when x overflows? • And we’ll focus on unit testing using the JUnit
framework

• Intermixing, with any luck, slides and a demo

59 60

15
 61 62

Does it even run and return? Running JUnit tests

• If getGreeting doesn’t run and return without throwing • There are many ways to run JUnit
an exception, it cannot meet the specification test method, test classes, and test
suites
JUnit tag “this is a test” @Test • Generally, select the method,
name of test public void test_NoException(){ class or suite and Run As >> • A failure is when the
Run getGreeting RandomHello.getGreeting(); JUnit Test test doesn’t pass –
JUnit “test passed” assertTrue(true); • A green bar says “all tests pass” that is, the oracle it
(doesn’t execute if } • A red bar says at least one test computes is incorrect
exception thrown) failed or was in error • An error is when
something goes
Tests should have A unit test is a (stylized) • The failure trace shows which wrong with the
descriptive (often program! When you’re writing tests failed and why program that the test
unit tests (and many other tests),
very long) names you’re programming!
didn’t check for (e.g.,
a null pointer
exception)
61 62

63 64

Don’t forget that

Eclipse can help you
Does it return one of the greetings? A JUnit test class get the right import
statements – use
Organize Imports
• If it doesn’t return one of the defined greetings, it import org.junit.*;
(Ctrl-Shift-O)
import static org.junit.Assert.*;
cannot satisfy the specification public class RandomHelloTest() {
@Test @Test
public void testDoes_getGreeting_returnDefinedGreeting() { public void test_ReturnDefinedGreeting() {
String rg = RandomHello.getGreeting(); …
for (String s : RandomHello.greetings) { }
if (rg.equals(s)) { @Test
assertTrue(true); public void test_EveryGreetingReturned() {
return; …
} }
} … ❑ All @Test methods run when
fail("Returned greeting not in greetings array"); } the test class is run
}
❑ That is, a JUnit test class is a
set of tests (methods) that
share a (class) name

63 64

16
 65 66

Does it return a random greeting? What about a sleazy developer?

if (randomGenerator.nextInt(2) == 0) {
@Test
public void testDoes_getGreetingNeverReturnSomeGreeting() { return(greetings[0]);
int greetingCount = RandomHello.greetings.length; } else
int count[] = new int[greetingCount]; return(greetings[randomGenerator.nextInt(5)]);
for (int c = 0; c < greetingCount; c++)
count[c] = 0; Run it 100
for (int i = 1; i < 100; i++) { times
String rs = RandomHello.getGreeting();  Flip a coin and select either a random or a specific greeting
for (int j = 0; j < greetingCount; j++)
if (rs.equals(RandomHello.greetings[j]))
 The previous “is it random?” test will almost always pass given this
If even one implementation
count[j]++;
} greeting is
for (int j = 0; j < greetingCount; j++) never  But it doesn’t satisfy the specification, since it’s not a random choice
if (count[j] == 0) returned,
fail(j+"th [0-4] greeting never returned");
assertTrue(true); it’s unlikely
} to be
random (1-
0.8100)

65 66

67 68

Instead: Use simple statistics A JUnit test suite

@Test import org.junit.runner.RunWith;
public void test_UniformGreetingDistribution() {
❑ Define one suite for each
import org.junit.runners.Suite;
// …count frequencies of messages returned, as in program (for now)
// …previous test (test_EveryGreetingReturned) ❑ The suite allows multiple
@RunWith(Suite.class)
@Suite.SuiteClasses({ test classes – each of
float chiSquared = 0f; which has its own set of
RandomHelloTest.class,
float expected = 20f;
SleazyRandomHelloTest.class @Test methods – to be
for (int i = 0; i < greetingCount; i++)
}) defined and run together
chiSquared = chiSquared +
((count[i]-expected)* public class AllTests { ❑ Add tc.class to the
(count[i]-expected)) // this class remains completely @Suite.SuiteClasses
// empty, being used only as a
/expected; annotation if you add a
if (chiSquared > 13.277) // df 4, pvalue .01 // holder for the above
// annotations new test class named tc
fail("Too much variance");
} } ❑ So, a JUnit test suite is a
set of test classes (which
makes it a set of a set of
test methods)

67 68

17
 69 70

JUnit assertion methods ArrayIntList: example tests

@Test @Test
…causes the current test to fail… public void testAddGet1() { public void testIsEmpty() {
fail() immediately ArrayIntList list = new ArrayIntList list = new
assertTrue(tst) if tst is false ArrayIntList(); ArrayIntList();
list.add(42); assertTrue(list.isEmpty());
assertFalse(tst) if test is true
list.add(-3); list.add(123);
assertEquals(expected, actual) if expected does not equal actual list.add(15); assertFalse(list.isEmpty());
assertSame(expected, actual) if expected != actual assertEquals(42, list.get(0)); list.remove(0);
assertEquals(-3, list.get(1)); assertTrue(list.isEmpty());
assertNotSame(expected, actual) if oracle == actual assertEquals(15, list.get(2)); }
assertNull(value) if value is not null }
assertNotNull(value) if value is null

• Can add a failure message: assertNull(“Ptr isn’t null", value)  High-level concept: test behaviors in combination
• expected is the oracle – remember this is the first (leftmost)  Maybe add works when called once, but not when call twice
param  Maybe add works by itself, but fails (or causes a failure) after calling remove
• The table above only describes when to fail – what happens if an
assertion succeeds? Does the test pass?

69 70

71 72

A few hints: data structures A few general hints

• Need to pass lots of arrays? Use array literals • Test one thing at a time per test method
public void exampleMethod(int[] values) { ... } • 10 small tests are much better than one large test
...
• Be stingy with assert statements
exampleMethod(new int[] {1, 2, 3, 4});
exampleMethod(new int[] {5, 6, 7}); • The first assert that fails stops the test – provides no
information about whether a later assertion would have failed
• Need a quick ArrayList?
List<Integer> list = Arrays.asList(7, 4, -2, 3, 9, 18); • Be stingy with logic
• Avoid try/catch – if it’s supposed to throw an exception, use
• Need a quick set, queue, etc.? Many take a list
Set<Integer> list = new HashSet<Integer>( expected= ... if not, let JUnit catch it
Arrays.asList(7, 4, -2, 9));

71 72

18
 73 74

Test case dangers More JUnit

• Dependent test order • Timeouts – don’t want to wait forever for a test to complete
• If running Test A before Test B gives different results from running • Testing for exceptions
Test B then Test A, then something is likely confusing and should @Test(expected = ArrayIndexOutOfBoundsException.class)
be made explicit public void testBadIndex() {
ArrayIntList list = new ArrayIntList();
• Mutable shared state
list.get(4); // this should raise the exception
• Tests A and B both use a shared object – if A breaks the object, } // and thus the test will pass
what happens to B?
• This is a form of dependent test order • Setup [teardown] – methods to run before [after] each test
• We will explicitly talk about invariants over data representations and case method [test class] is called
testing if the invariants are ever broken

73 74

75 76

Content 3. Integration test

1. Testing overview • Examine the interface between modules as well as the
input and output
2. Unit Test • Stub/Driver:

3. Integration Test • A program that simulates functions of a lower-level/upper-

level module

75 76

19
 77 78

3.1. Top-down approach 3.2. Bottom-up approach

• Defects based on misunderstanding of specification can be • Lower modules are independent => test independently and
detected early on a parallel
• Effective in newly developed systems • Effective in developing systems by modifying existing
• Need test stubs (can be simply systems
returning a value) • Need test drivers (more complex with controlling)

77 78

79 80

3.3. Other integration test techniques 3.4. Regression test

• Big-bang test “When you fix one bug, you introduce several new bugs”
• Wherein all the modules that have completed the unit • Re-testing an application after its code has been modified
tests are linked all at once and tested to verify that it still functions correctly
• Reducing the number of testing procedures in small- • Re-running existing test cases
scale program; but not easy to locate errors • Checking that code changes did not break any previously working
functions (side-effect)
• Sandwich test
• Run as often as possible
• Where lower-level modules are tested bottom-up and
higher-level modules are tested top-down • With an automated regression testing tool

79 80

20
81

21