Testing
Testing
Simple Tests
Most teams I interact with today write tests of some kind. They are often using
tools such as nunit/junit and building small automated tests over their code.
Most teams have derived value from the introduction of automated testing.
Automated testing will find bugs. Automated testing will find bugs in a faster
way and at a lower cost than only doing manual testing on nearly any piece
of software. You can find some outliers to this and it is kind of a fun exercise
to go through in your head/in a discussion with a few others but as a rule
the statements which have been made about automated testing having these
properties are true.
One of the keys of automated testing is that doing it tends to find bugs sooner.
Having an automated test which tells me fifteen seconds after I make a code
change that the change broke some other behaviour in my implementation of
it is far cheaper than me finding out three days later from people testing the
change as a whole.
A term commonly used to describe this aspect is “fail fast”. A main goal of
automated testing is to find mistakes as quickly as possible. The longer a bug is
in code the more things that may become dependent upon it either directly or
indirectly. Some bugs after years of being in a system become weeks of effort to
1
remove, not because the bug itself is so hard to remove but because there have
been over time a hundred other things which rely upon the buggy behaviour.
Writing even simple tests is better in most circumstances than writing no tests.
There are drawbacks in how they are structured but it is still better than . . .
nothing. Let’s look at a small piece of code and how to structure a simple test
for it.
public class InventoryItem : Aggregate {
bool _activated = true;
string _deactivationReason;
2
object. The problem with putting public state on your domain objects is that
once it is there . . . people will use it in other places as well often where they really
should not be using it. The putting of public state is breaking the encapsulation
of the object.
I have seen teams jump through hoops to avoid this. As example they would
put a rule into something like jdepend/ndepend to not allow the usage of the
properties inside the domain model itself but only within the tests. This works
but . . . it is kind of annoying to have as a system. It would be much better to
not have the getters at all, it is rather difficult for people to misuse things that
do not exist.
Perhaps surprisingly events can come into the equation as an answer to this
issue. What if the InventoryItem were instead to support events?
[TestFixture]
public class InventoryItemTests {
[Test]
public class DeactivatingAnActiveItemMakesItDeactivated() {
var item = new InventoryItem(13, "pickle ice cream");
item.Deactivate("some reason");
var event = item.GetUncommittedEvents().Single();
var typed = Assert.IsType<Deaactivated>(event);
Assert.AreEqual(13, typed.Id);
Assert.AreEqual("some reason", typed.DeactivationReason);
}
}
The asserts now are not checking the InventoryItem itself but are happening on
the events that the InventoryItem produced. All of the state of the InventoryItem
is completely private. The test nor any of the other code has the foggiest idea
what the internals of an InventoryItem look like. They cannot possibly couple
to it unless they start doing some really wild things like using private reflection
which should raise a red flag in any code review.
If you wanted to refactor the InventoryItem so long as you put out the same
events it would not break anything. Anyone who has done a large refactor of
a domain object and watched 200 tests break due to something silly should be
able to appreciate the value of not exposing the internals of the domain object.
A huge number of tests are written in precisely this manner. I want to be clear
that there is nothing wrong with writing tests such as this. These tests are fine
grained, understandable, and generally manageable. They are absolutely unit
tests and do not even border on possibly being integration tests but there is still
a lot more value that can be derived from tests.
What if we wanted to put a bit more structure to things to try to get some
further information out of the test fixtures? To start with let’s just try providing
some structure and language around them. Instead of writing independent tests,
3
let’s move to a fixture.
Test Base
Test Bases are a very common pattern to be looking at. Instead of dealing with
a method per test the move is going to be a class per test. The focus is going to
be on moving to a Given/When/Then structure. Assertions are going to end up
with one per method where the method name is describing what that assertion
actually is.
One of the main reasons for moving to a Test Base is because it contains a lot
more metadata associated with the test. Whether the tests are being run in
development, on a CI server, or as part of acceptance etc, the tests come out in
a much more “palatable” way for non-technical people to be able to look at the
results.
What is being sought after here is more of a “Specification by Example” though
there are many terms which have been used in varying works ovr the years. It is
mainly breaking apart the test into multiple pieces that have “data” associated
with them so that data can be pulled out and shown to people.
The simplest of these is to start with naming.
Instead of writing a test such as
[TestFixture]
public class InventoryItemTests {
[Test]
public class DeactivatingAnActiveItemMakesItDeactivated() {
var item = new InventoryItem(13, "pickle ice cream");
item.Deactivate("some reason");
var event = item.GetUncommittedEvents().Single();
var typed = Assert.IsType<Deaactivated>(event);
Assert.AreEqual(13, typed.Id);
Assert.AreEqual("some reason", typed.DeactivationReason);
}
}
This same test could be written in the form of
[TestFixture]
public class deactivating_an_active_item_makes_it_deactivated {
InventoryItem item;
List<Event> events = new List<Event>();
[SetUp]
public void Setup() {
item = new InventoryItem(13, "pickle ice cream");
4
item.Deactivate("some reason");
events.Add(item.GetUncommitedEvents());
}
[Test]
public void one_event_is_produced() {
Assert.AreEqual(1, events.Count);
}
[Test]
public void the_event_is_a_deactivation_for_the_correct_item() {
Assert.AreSame(new InventoryItemDeactivated(13, "pickle ice cream"), events[0]);
}
}
This is an equivalent test to the first test which was written but there are some
differences. What would happen if you ran this in your ide? Most ides when
bringing up a test runner will show the name of the fixture then under the fixture
but indented the name of each test with it being red/green for pass/fail.
+deactivating_an_active_item_makes_it_deactivated |–> one_event_is_produced
|–> the_event_is_a_deactivation_for_the_correct_item
Could a business person read this?
What if we went a really really difficult step past this and replaced the underscores
with space as it was output?
• deactivating an active item makes it deactivated |–> one event is produced
|–> the event is a deactivation for the correct item
This is beginning to look like something they can pretty easily interact with . . .
but . . . this is just the beginning. This works fine for unit tests in general but
for tests of domain objects we can do a lot better than this.
The problem which will be run into using the current style of testing should be
apparent when looking at two of them.
[TestFixture]
public class deactivating_an_active_item_makes_it_deactivated {
InventoryItem item;
List<Event> events = new List<Event>();
[SetUp]
public void Setup() {
item = new InventoryItem(13, "pickle ice cream");
item.Deactivate("some reason");
events.Add(item.GetUncommitedEvents());
}
5
[Test]
public void one_event_is_produced() {
Assert.AreEqual(1, events.Count);
}
[Test]
public void the_event_is_a_deactivation_for_the_correct_item() {
Assert.AreSame(new InventoryItemDeactivated(13, "pickle ice cream"), events[0]);
}
}
[TestFixture]
public class reactivating_an_active_item_makes_it_active_again {
InventoryItem item;
List<Event> events = new List<Event>();
[SetUp]
public void Setup() {
item = new InventoryItem(13, "pickle ice cream");
item.Deactivate("some reason");
item.Reactivate("foo");
events.Add(item.GetUncommitedEvents());
}
[Test]
public void one_event_is_produced() {
Assert.AreEqual(1, events.Count);
}
[Test]
public void the_event_is_a_deactivation_for_the_correct_item() {
Assert.AreSame(new InventoryItemReactivated(13, "some reason"), events[0]);
}
}
OK so now there are two of them. How much of what is there is . . . duplication?
What will happen when a third or a twenty seventh is added? Quite minimal
amounts of code are changing from scenario to scenario.
Much of this code which is duplication but not necessarily direct duplication can
be factored out of the tests while still keeping the same structure and ability to
run. Some examples of this would be the creation of the domain object either
through events or just an instantiation, the getting of the events from the domain
objct, the declaration of the SUT, the declaration of the List to hold the events
. . . if you went on a line basis there are nine lines of code in that example. Of
those nine lines of code six are going to be duplicated (not counting {} but
counting the attribute).
6
Getting rid of this duplication would almost certainly be a good idea.
Let’s try to get rid of the Setup() in the test class and move it to a GWT
(Given/When/Then) structure.
[TestFixture]
public class MyTestBase {
protected DomainObject sut;
public abstract void Given();
public abstract void When();
[SetUp]
public void Setup() {
Given();
When();
}
}
So now there is a TestBase class which has a SUT (subject under test) and
abstract definitions for a method called Given() and When(). It will call these
methods during the SetUp phase of the test.
public class when_deactivating_an_inventory_item : MyTestBase {
public override DomainObject Given() {
return new InventoryItem(12, "some product");
}
[Test]
pulic void the_item_is_deactivated() {
Assert.That(produced.OnlyContains<InventoryItemDeactivated>(x=>x.InventoryItemId
}
}
This has a lot less duplication in it which isn’t too shabby it being that we did
that by writing an . . . eleven line . . . file.
Many people will stop here. At this point there is a reasonable structure to the
tests, basic information can be gotten out of them and turned into something
others can consume, and things are relatively simple. But . . . we can do more.
Actual Documentation
So if you have already reached the point of the code above, you will quickly find
that just using the method names while useful leaves more than just a little bit
7
to be desired.
It gives someone a very high level overview of what that test is . . . likely . . .
doing but there will still be questions about what it is . . . actually . . . doing.
Beyond this there will be times that just the naming is not quite clear even for
people who know the domain fairly well. You could teach them to read code
. . . to be fair it is not that hard to teach them and some already have at least a
basic idea of how code works. There are however other ways.
What is missing in the documentation produced is essentially all of the relevant
information about what is involved. It is just a series of names. What if further
information were also to be provided?
Looking again at the base fixture you can see that there is a method Given()
which returns the subject under test for every test.
[TestFixture]
public class MyTestBase {
protected DomainObject sut;
public abstract void Given();
public abstract void When();
[SetUp]
public void Setup() {
Given();
When();
}
}
What if you were to call that method to get the domain object as part of printing
your documentation?
• deactivating an active item makes it deactivated |–> SUT InventoryItem
{id=12, productName=“some product”, activated=true} |–> one event is
produced |–> the event is a deactivation for the correct item
Getting the text representing the object could be done in a few ways. The first
would be to just put a ToString() on the domain object. The second would be
to write a little bit of code using reflections etc that can look at the state of the
object and print it out. You might also consider having the domain object return
another object that represents it’s state. You might even instead of producing a
string produce a string of json by serializing it.
There is not one method which is right here it depends how many domain
objects you have, what formats people involved may understand, how important
standardized printing is, and how much flexibility you want with the printing.
All of these methods are however quite similar in how they operate.
Once this is done however the test is much more useful for someone to read but
this process can go significantly further!
8
The next thing that would probably want to be done is to get the operation
being done included in the documentation. You know . . . that whole “When”
thing. I hear it is kind of important in terms of understanding what a test
actually does.
Currently the “When” is calling a method. It is feasible that we could write
code that parses the IL/byte code of the When() method seeing which method
is being called on the domain object but this sounds . . . complicated.
There is however another way of “calling a method” on a domain object. That
is if you squint at it right what a Command does. What if instead of calling the
method directly instead a Command were passed down which was then pushed
through a CommandHandler that would call the behaviour on the domain object?
Eek! This is starting to sound less and less like a unit test! Let’s however take a
look at one before getting too upset over what might be occurring.
[TestFixture]
public class Specification<T> {
protected T sut;
public abstract T Given();
public abstract Command When();
[SetUp]
public void Setup() {
Given();
OnHandler(When());
}
}
This base class could then be used in a derived class as follows.
public class when_deactivating_an_inventory_item : Specification<InventoryItem> {
public override InventoryItem Given() {
return new InventoryItem(12, "some product");
}
[Test]
pulic void the_item_is_deactivated() {
9
Assert.That(produced.OnlyContains<InventoryItemDeactivated>(x=>x.InventoryItemId
}
}
But more importantly let’s try running the document generator against this
class.
• when deactivating an inventory item |–> SUT InventoryItem {id=12,
productName=“some product”, activated=true} |–> Inventory Item De-
activated {id=12, reason=“foobaz”} |–> one event is produced |–> the
event is a deactivation for the correct item
We will also commonly print out the SUT after the test has been run. This
would result in
• when deactivating an inventory item |–> SUT InventoryItem {id=12,
productName=“some product”, activated=true} |–> Inventory Item Deac-
tivated {id=12, reason=“foobaz”} |–> results with InventoryItem {id=12,
productName=“some product”, activated=false} |–> one event is produced
|–> the event is a deactivation for the correct item
Obviously I am using a quite terse form of “printing” here to save vertical space.
You can format these in different ways. I tend to make them one page each.
This is so each is on a page by itself which is rather useful when passing around
pieces of paper with them on it.
As a quick example if I were dealing with stock market orders I would show the
order flow and then print up what the resulting order book was. Then I would
say what was to happen and show the result (eg: Order Placed {id=. . . .}) and I
would show what the new order book is after the operation succeeded. I would
format the order book to look at much like whatever trading station they use as
I could. Reading this would be completely natural to a trader.
Using one page each also allows you to do nice things like use vertical space for
describing the domain object. Most tests should easily fit on a single page and
having them as single pages makes them far easier to “move around” them as
pieces of paper.
Another thing which can be considered is the setting up of a process around
these pieces of paper where they go through approvals as part of the feature
being completed. This “forces” interaction of the business side in terms of saying
“the system is doing the right thing”.
It is important to note that the creation of these printouts from tests is more a
“let’s hang out on Friday night and build it while having some pizza” as opposed
to anything vaguely resembling an actual project. You should in an evening be
able to spike something which functions. If it turns out two months later that
something else is prefered for certain use cases, why not make that one too? If
the cost is a case of beer and three or four pizzas it is a relatively small expense.
10
Oh, and if you are a manager reading this . . . don’t cheap out . . . get good
pizza and beer :-D
As another example of something to do on a “pizza evening” if you look at the
tests the Given() returns a series of events. What if you were to build a . . . graph.
You could then easily find all of the tests quickly which were associated to a given
event. Building up such a visualization would not take a long time, it is basically
just going through all of the tests, get the command/aggregate/events/etc and
putting up nodes/connections into the graph. This can in fact be a quite useful
high level visualization of your system. In terms of actually producing it, it
might take a Saturday afternoon.
Closing
There has been a lot which has been gone through in this chapter. The key thing
to take away however is that testing is a crucial aspect of building/maintaining
a system. Testing is not only important for verifying the system is functioning
properly but also can be a mechanism for ensuring that it is doing the right
thing.
You may be scratching your head as those two things sound identical they are
not. A system can easily be “functioning properly” but not “doing the right
thing”. This is one of the largest problems in computer science.
Communicating what the system actually does is a challenge for many teams.
Using tests as a mechanism for communication / documentation of the system
is generally a good idea.
Tests can become a primary form of communication / documentation of the
system. This sounds unusual at first but it actually works quite well. One of the
main benefits of heading down this road is the the “specifications” of the system
contained in the tests can actually execute against the system. It is not that
you think something might be supported, it is executable, it either does or it
does not.
Being certain about things has a significantly more than non-zero value associated
with it.
11