Unit Testing

Prepared by:
Priya Sharma Trivedi
Agenda

 Introduction
 The need of testing and it’s Terminology
 Unit Testing
 Right B.I.C.E.P.s.?
 Benefits of Unit Testing
 Naming standards for unit tests
The Problem
A vicious cycle – the more pressure a you (Developer) feel, the fewer
tests you writes. The fewer tests you write, the less productive you are
and the less stable your code becomes. The less productive and
accurate you are, the more pressure you feel.

Result of this problem

 error - a mistake made by a human (in a software development
activity)
 defect (or fault) - the result of introducing an error into a software
artifact (SRS, SDS, code, etc.)
 failure - a departure from the required behavior for a system
Unit Testing
 Unit testing is a software verification and validation method in which a
programmer tests if individual units of source code are fit for use.
 The goal of unit testing is to isolate each part of the program and show that
the individual parts are correct.
 Unit Testing checks that behaves correctly.
 Static Testing
 testing a unit without executing the unit code
 Dynamic Testing
 testing a unit by executing a program unit using test data

Some common definitions of a unit:

 A unit is the smallest testable part of an application.
 In procedural programming a unit may be an individual function or
procedure.
 The smallest chunk that can be compiled by itself
 A stand alone procedure of function
 Something so small that it would be developed by a single person
Why Unit Testing????????????

 Without unit tests defects show up in the end of development

 Finding them is difficult. Removing them time-consuming and Complex

 With unit tests we can find defects much sooner and easier, continuously
removing them before adding new features

 We can also check if new features break our code

 Unit Test:  Important code should have unit tests
 Tests should be written together with the code
Unit-tests  Code should pass minimum 70% of the unit tests
before it can be released
Should validate expected functionality
at the level of:
• individual class
• small groups of collaborating classes
CODE Automatic
• work package
software test
run process
Code Documentation

Software
developer
Unit-test Component

• Test execution framework

• Testing naming and structure
Test
• Test case specification template programs
Unit-test: Test frameworks
Aim: to help developers: • to produce code for unit-testing
• to run tests in automatic way

Our constrains: Our inputs:

•TIME •What is available as free open source
code.

What we propose:
JUnit
NUnit
SimpleTest

What we are doing:

•Trying Unit Testing Tools Our plans:
in the our projects. • Implementation of the Unit Testing
• Preparing “HowTo” documents to Procedure in the our project
make easier the installation of these environment.
tools and the start with process
Unit Testing makes your developer lives easier
 Easier to find bugs
 Easier to maintain
 Easier to understand
 Easier to Develop

You have already done Unit testing

 Not structured
 Not Repeatable
 Not on all your code
 Not easy to do as it should be

 A framework is missing
Designing unit tests

Type of Unit Test

Positive Test Negatives

Test the function Test the function

with valid input data With invalid data
 When to write the test
“Whenever you are tempted to type something into a print
statement or a debugger expression, write it as a test
instead.”...

 During Development - When you

need to add new functionality to the
system, write the tests first. Then you
will be done developing when the
test runs.

 During Debugging - When

someone discovers a defect in your
code, first write a test that will
succeed if the code is working. Then
debug until the test succeeds.
 (subprogram,
object class,
package, module)
An individual
program unit Test for both success
Pre-conditions:
& failure.
Those things that
For boundary
must be true
conditions
before you run
(Eg. The object’s buffer For general
has been initialized) the code.
functionality Etc..

What should be Databases,

tested ? Stored
Post-conditions: Procedures
things that must , Triggers,
be true after you Tables,
Database
run the code. Indexes
conversion
(Eg. method will never NT
.OCX,
return null) Services
Invariants: .DLL,
things that must .EXE and
always be true. other
binary
(Eg. The list size will formatted
always be greater than or executables
equal to 0)
Example of How To Think of Tests

 The required functionality for a particular service

(getVersionsForProductId) is the ability to get a list of all versions
for a given product id. If there’s no product for that id, then an
exception is thrown. If there’s no versions for a valid product id,
then an empty list is returned.

The Tests That Are Needed

Starting from the easiest to the hardest:
 If there’s no product for that id, then an exception is thrown (pre-
condition)
 If there’s no versions for a valid product id, then an empty list is
returned (post-condition)
 If there are versions for a product id, then a non-empty list of all the
versions is returned (post-condition)
 Unit Testing Tasks and Steps:

 Step 1: Create a Test Scenario

 Step 2: Create Test Cases and Test Data
 Step 3: If applicable create scripts to run test cases
 Step 4: Once the code is ready execute the test cases
 Step 5: Fix the bugs if any and re test the code
 Step 6: Repeat the test cycle until the “unit” is free of all bugs
Testing Steps
What Makes a Good Unit Test?
 It sufficiently tests the contract. It doesn’t need to be complete, just
“sufficient.”
 If it’s complex code, then have lots of tests.
 If it’s a simple accessor, then you don’t need a test.
 Most things are in between…

 It runs quickly. Part of the point is to make it so that you’re able to run them
regularly, so speed plays into that.
 It’s much easier to fix a bug you introduced five minutes ago than one
you did five weeks ago…

 Tests are independent.

 Loosely coupled functionality enables independent tests.
 Write loosely coupled, highly cohesive code!

 Run the tests regularly.

 You can be “fearless” when you need to make changes, because you
don’t have to worry that you might’ve broken something.
 When to Run Tests
 When you write a new method……compile and run local unit tests
 When you fix a bug……run the test that illustrates that bug.
 Any successful compile……run local unit tests.
 Before you check in……run all tests.
 Continuously…...check out and build the project from scratch including
all unit tests.

Do not check in code that…

 Is incomplete (e.g. missing dependencies)
 Doesn’t compile
 Compiles but breaks other code
 Doesn’t have unit tests
 Has failing unit tests
 Passes its tests but causes other tests to fail.
 Test Case Sample

Test Case Test Case TC Input Expected Actual Result Pass / Remarks
Purpose Procedure Data Result Fail

ID which can What to How to Input What What actually Pass / Comment
be referred to Test Test Data should happens. This Fail if any
in other docs happen? column can
like “TM” be omitted
“Root Cause when defect
Analysis of recording tool
defects etc is used.

Additionally the following information may also be captured:

a) Unit Name and Version Being tested
b) Tested By
c) Date
d) Test Iteration (One or more iterations of unit testing may be performed)
 Steps to Effective Unit Testing

Documentation: Early on document all the Test Cases needed to test your
code. Document the Test Cases, actual Results when executing the Test
Cases, Response Time of the code for each test case.
Important advantages if the test cases and the actual execution of test
cases are well documented.
 Documenting Test Cases prevents oversight.
 Documentation clearly indicates the quality of test cases.
 If the code needs to be retested we can be sure that we did not
miss anything.
 It provides a level of transparency of what was really tested during
unit testing. This is one of the most important aspects.
 It helps in knowledge transfer in case of employee attrition.
 Sometimes Unit Test Cases can be used to develop test cases for
other levels of testing
What should be tested when Unit Testing
It could be a screen or a component or a web
service

Test Cases to verify all the screen Test cases to verify the
elements that need to appear on the spelling/font/size of all the “labels” or
screens. text that appears on the screen.

Test Cases such that every line of code Test Cases such that every condition in
in the unit is tested at least once in a case of “conditional statements” is
test cycle. tested once.

Test Cases to test the

Test Cases to verify how various errors
minimum/maximum range of data that
are handled.
can be entered.

Test Cases to verify if all the

validations are being performed
 Write the
test first

Never write
a test that Use mock
succeeds the Objects
1st time

Charles'
Six Rules
of Unit
Testing
Start with
Loose
null case, or
coupling &
something
testability go
that doesn’t
hand in hand
work
Try
something
trivial to
make the test
work
 Unit Testing Guidelines

 Keep unit tests small and fast  Think white-box

 Unit tests should be fully  Test the trivial cases too
automated and non-interactive  Focus on execution coverage first
 Make unit tests simple to run  Cover boundary cases
 Measure the tests  Provide a random generator
 Fix failing tests immediately  Test each feature once
 Keep testing at unit level  Use explicit asserts
 Start off simple  Provide negative tests
 Keep tests independent  Design code with testing in mind
 Keep tests close to the class  Don't connect to predefined
being tested external resources
 Name tests properly  Write tests to reproduce bugs
 Test public API  Know the limitations
 Think black-box
Are we testing the Right B.I.C.E.P.s.?
Testing without a strategy is futile.

R Are the results right?

B Are all the Boundary conditions correct?
I Can you test the Inverse relationship?
C Can you Cross-check using other methods/means?
E Can you force Error conditions to happen?
P Are Performance conditions within bounds?
s Always go for the whole set.
Right-BICEP

Right: Are the results right?

 Validate results

 Does the expected result match what the method does?

 If you don’t know what “right” would be, then how can you test?
How do you know if your code works?
 Perhaps requirements not known or stable

 Make a decision. Your tests document what you decided.

Reassess if it changes later.

Right-BICEP
Boundary Conditions
 Garbage input values

 Badly formatted data like ’1970-23-12’ when 1970-12-23 is required.

 Empty or missing values (0, null, etc.)

 Values out of reasonable range

 Duplicates if they’re not allowed

 Unexpected orderings

 Totally bogus or inconsistent values, such as filenames.

 Validity like age in years between 0 and 90 e.g. 115.

 Things that arrive out of order.

Right-BICEP
Check Inverse Relationships
 If your method does something that has an inverse, then apply the

inverse
 E.g. square and square-root. Insertion then deletion.
 Beware errors that are common to both your operations

 Seek alternative means of applying inverse if possible

Right-BICEP

Are the boundaries C.O.R.R.E.C.T.?

C Conformance. Is the format Ok?
O Ordering. Is the set (un)ordered as appropriate e.g. as dictated by
invariant.
R Is the value within acceptable Range?
R Does the code Reference anything external not under its direct control?
E Does the value Exist?
C Countability; Are there enough values?
T Tim(e|ing). Is everything happening in the right order?
Right-BICEP

Cross-check using other means

 Can you do something more than one way?
 Your way, and then the other way. Match?

 Are there overall consistency factors you can check?

 Overall agreement

If e.g. you are devising a square root for an embedded system without float capability,
you could test it (in the host environment) using the standard implementation, like
so:
public void testSquarerootUsingStd( ) {
double number=3830900.0;
double root1= mySquareRoot ( number ) ;
double root2= Math.sqrt ( number ) ;
assertEquals ( root2 , root1 , 0.0001 ) ;
}
 Right-BICEP

Force Error Conditions; Can your module survive?

 Strange things in time keeping (like a clock that has been set back or
summer and winter time).
 Failures outside your code:
 Invalid parameters, out of range values, etc.

 Out of memory, disk full, network down, etc.

 Can simulate such failures

 High system load.

 Limited color availability in user interface.

 Very high or very low screen resolution.

Right-BICEP

 Performance
A normal unit test should test for the functional requirements. But if the
non-functional requirement performance or speeds depend on the size of
the data, you should test that too.
 Perhaps absolute performance, or

 Perhaps how performance changes as input grows.

 Perhaps separate test suite in JUnit

Example: a spam filter that tests against “naughty” or black-listed mail

servers. Test your code with different lists sizes. Typically a small list for
build testing, but maybe also with a longer list e.g. once a day.
 Benefits of Unit Testing

It provides a strict, written contract that It find problems early in the development
the piece of code must satisfy. cycle.

It allows the programmer to refactor code It may reduce uncertainty in the units
at a later date, and make sure the module themselves and can be used in a bottom-up
still works correctly (i.e. regression testing). testing style approach.

It provides a sort of living documentation of By testing the parts of a program first and
the system. Developers looking to learn
what functionality is provided by a unit and then testing the sum of its parts, integration
how to use it. testing becomes much easier.
Result
 # of Test Cases (Test Development Phase)
 # of Test Cases Executed (Test Execution Phase)
 # of Test Cases Passed (Test Execution Phase)
 # of Test Cases Failed (Test Execution Phase)
 # of Test Cases Under Investigation (Test Development Phase)
 # of Test Cases Blocked (Test dev/execution Phase)
 # of Test Cases Re-executed (Regression Phase)
 # of First Run Failures (Test Execution Phase)
 Total Executions (Test Reporting Phase)
 Total Passes (Test Reporting Phase)
 Total Failures (Test Reporting Phase)
 Test Case Execution Time ((Test Reporting Phase)
 Test Execution Time (Test Reporting Phase)
Result…Cont……
 % Complete
 % Defects Corrected
 % Test Coverage
 % Rework
 % Test Cases Passed
 % Test Effectiveness
 % Test Cases Blocked
 % Test Efficiency
 1st Run Fail Rate
 Defect Discovery Rate
 Overall Fail Rate
Naming standards for unit tests

Test name should express a specific requirement. The Public void

basic naming of a test comprises of three main parts Sum_simpleValues_Calculated ()
[MethodName_StateUnderTest_ExpectedBehavior]

Test name should include the expected input or state and Public int Sum(params int[]
the expected result for that input or state values), Public int
Sum_NumberIsIgnored()
Test name should be presented as a statement or fact of Public void
life that expresses workflows and outputs SumNegativeNumber2()

Test Name should only begin with Test if it is required by testCalculator()

the testing framework or if it eases development and
maintenance of the unit tests in some way.

Test name should include name of tested method or class. Public void
[MethodName_StateUnderTest_ExpectedBehavior] Sum_NegativeNumAs1stParam_E
xcepThrown()
Variable names should express the expected input and BAD_DATA or EMPTY_ARRAY or
state NON_INITIALIZED_PERSON
Naming standards for unit tests- Cont……

Target / Subject to refer to the piece of functionality that is testing.

Fixture Synonymous with "TestFixture", a fixture is a class that contains a set of
related tests.
Suite Test Suites are an older style of organizing tests. They're specialized
fixtures that programmatically define which Fixtures or Tests to run.
Test Methods within the Fixture that are decorated with the [Test] attribute and
contain code that validates the functionality of our target.
SetUP Test Fixtures can designate a special piece of code to run before every Test
within that Fixture. That method is decorated with the [Setup] attribute.
TearDown a method with the [TearDown] attribute is called at the end of every test
within a fixture.
Fixture Setup Similar to constructors.

Fixture Similar to finalizers

TearDown
Category : The [Category] attribute when applied to a method associates the Test
within a user-defined category.
Ignore: Tests with the [Ignore] attribute are skipped over when the Tests are run.
Explicit: Tests with the [Explicit] attribute won't run unless you manually run them.
Naming standards for unit tests- Cont……

CONSIDER: Separating your Tests If you have a requirement where you want to test in
from your Production Code. production or verify at the client's side, you can
accomplish this simply by bundling the test library with
your release.
CONSIDER: Deriving common Fixtures In scenarios where you are testing sets of common
from a base Fixture classes or when tests share a great deal of duplication,
consider creating a base TestFixture that your Fixtures
can inherit.
CONSIDER: Using Categories instead Suites represent significant developer overhead and
of Suites or Specialized Tests maintenance. Categories offer a unique advantage in the
For example, you could execute only UI and at the command-line that allows you to specify
"Stateful" tests against an environment to which categories should be included or excluded from
validate a database deployment. execution.
CONSIDER: Splitting Test Libraries (You can always go back)
into Multiple Assemblies
AVOID: Empty Setup methods you should only write the methods that you need today.
Adding methods for future purposes only adds visual
noise for maintenance purposes
Naming standards for unit tests- Cont……
DO: Name Tests after Functionality The test name should match a specific unit of
For example, a test with the name functionality for the target type being tested.
CanDetermineAuthenticatedState provides more Some key questions you may want to ask
direction about how authentication states are yourself: "what is the responsibility of this class?"
examined than Login. "What does this class need to do?" Think in terms
of action words.
DO: Document your Tests Most tests require special knowledge about the
A few comments here and there are often just the functionality your testing, so a little
right amount to help the next person understand documentation to explain what the test is doing is
what you need to test and how your test helpful.
approaches demonstrates that functionality.
CONSIDER: Use "Cannot" Prefix for Expected Since Exceptions are typically thrown when your
Exceptions application is a performing something it wasn't
Examples: CannotAcceptNullArguments, designed to do, prefix "Cannot" to tests that are
CannotRetrieveInvalidRecord. decorated with the [ExpectedException] attribute.

CONSIDER: Using prefixes for Different If your application has features that differ slightly
Scenarios for application roles, it's likely that your test
Some have adopted a For<Scenario> syntax names will overlap.
(CanGetPreferencesForAnonymousUser). Other
have adopted an underscore prefix _<Scenario>
(AnonymousUser_CanGetPreferences).
Naming standards for unit tests- Cont……

AVOID: Ignore Attributes with no Tests that are marked with the Ignore attribute
explanation should include a reason for why this test has been
disabled.
AVOID: Naming Tests after If you find that your tests are named after the
Implementation methods within your classes, that's a code smell
If you changed your method name, would that you're testing your implementation instead of
the test name still make sense? your functionality.
AVOID: Using underscores as word- PascalCase should suffice. Imagine all the time you
separators save not holding down the shift key.
use_underscores_as_word_separators_fo
r_readability,
AVOID: Unclear Test Names Sometimes we create tests for bugs that are caught
it's important to avoid giving them vague late in the development cycle, or tests to
test names that represent a some external demonstrate requirements based on lengthy
requirement like FixForBug133 or requirements documentation. As these are usually
TestCase21. pretty important tests (especially for bugs that
creep back in).
Naming standards for unit tests- Cont……

Categories Using Categories is a powerful way to

dynamically separate your tests at runtime,
DO: Limit the number of Categories
however their effectiveness is diminished when
developers are unsure which Category to use.

CONSIDER: Defining Custom Category As Categories are sensitive to case and spelling,
Attributes you might want to consider creating your own
Category attributes by deriving from
CategoryAttribute.

Finished with Unit Testing

Summary

“Unit Testing” is the most important one. Detecting and fixing bugs early
on in the Software Lifecycle helps reduce costly fixes later on. The Above
article explains how Unit Testing should be done and the important points that
should be considered when doing Unit Testing.