Automatically writing Java unit tests with

Diffblue Cover
A Tutorial

Peter Schrammel
Outline
• Diffblue Cover: what it is and how it works
• Using Diffblue Cover
• when refactoring code

• when implementing new features

• together with TDD

• Q&A
What is
Diffblue Cover?
IntelliJ
> Preferences
> Plugins
> Marketplace
> Search for “Diffblue Cover”
> Install Community Edition
https://plugins.jetbrains.com/plugin/14946-diffblue-cover
What is Diffblue Cover?
1. git clone
https://github.com/Diffblue-benchmarks/Spring-petclinic
2. cd Spring-petclinic
3. git checkout a-test-intro
@Service 4. Open project in IntellIJ
public class CloudStorageService { 5. Right-click on CloudStorageService > Write tests

@Autowired
private AmazonS3 s3client;

public PutObjectResult uploadFileToBucket(String bucketName, String key, File file) {

return s3client.putObject(bucketName, key, file);
}

public S3Object downloadFileFromBucket(String bucketName, String key) {

return s3client.getObject(bucketName, key);
}

}
 What is Diffblue Cover?
1. git clone
https://github.com/Diffblue-benchmarks/Spring-petclinic
2. cd Spring-petclinic
@ExtendWith(SpringExtension.class) 3. git checkout a-test-intro
public class CloudStorageServiceTest { 4. Open project in IntellIJ
@MockBean
private AmazonS3Client amazonS3Client;
5. Right-click on CloudStorageService > Write tests

@Autowired
private CloudStorageService cloudStorageService;

@Test
public void testDownloadFileFromBucket() throws UnsupportedEncodingException {
StringInputStream objectContent = new StringInputStream("Lorem ipsum dolor sit amet.");
S3Object s3Object = new S3Object();
s3Object.setObjectContent(objectContent);
when(this.amazonS3Client
.getObject(or(isA(String.class), isNull()), or(isA(String.class), isNull())))
.thenReturn(s3Object);
assertSame(s3Object, this.cloudStorageService.downloadFileFromBucket("bucket-name", "key"));
}
....
What does it do?
JUnit
Java Java Create tests
source Compile byte unit source
code code tests code
What does it do?
JUnit
Java Java Create tests
source Compile byte unit source
code code tests code

● Tries to trigger execution of paths through a method

● Tries to use inputs that are similar to what a developer would choose
● Mocks dependencies that should be mocked
● Figures out what useful assertions may be
● Assertions always reflect current behavior
What does it do?
● Tries to trigger execution of paths through a method
● Tries to use inputs that are similar to what a developer would choose
● Mocks dependencies that should be mocked
● Figures out what useful assertions may be
● Assertions always reflect current behavior

What it doesn’t do…

● It doesn’t guess the intended behavior.
● It doesn’t replace all manual test writing.
● Does not create end-to-end (e.g. UI) tests.
 Currently supports
How should I use it? ● Ubuntu 18-20, macOS 10.15,
Windows 10
● IntelliJ CE/Ultimate 2019.3+
● Java 8 and 11
• Unit regression testing ● JUnit 4.7+ and 5
● Mockito 2.1+
• Speeding up unit test writing ● Spring 5+ / Spring Boot 2+
• Finding bugs by review https://docs.diffblue.com

Use cases we’ll look into:

• I’m going to do a refactoring

• I’m implementing a new feature

• I’m using TDD

 Desirable properties:
What is a unit test? •
•
Runs fast (a few ms)
Has no side effects on
other tests

@MockBean
private AmazonS3Client amazonS3Client;
@Test
public void testDownloadFileFromBucket() throws UnsupportedEncodingException {
StringInputStream objectContent = new StringInputStream( "Lorem ipsum dolor sit amet.");
S3Object s3Object = new S3Object();
s3Object.setObjectContent(objectContent); Arrange inputs and mocks
when(this.amazonS3Client
.getObject( or(isA(String. class), isNull()), or(isA(String. class), isNull())))
.thenReturn(s3Object);
S3Object actualS3Object = this.cloudStorageService.downloadFileFromBucket( "bucket-name", "key")
assertSame(s3Object, actualS3Object);
}
 Desirable properties:
What is a unit test? •
•
Runs fast (a few ms)
Has no side effects on
other tests

@MockBean
private AmazonS3Client amazonS3Client;
@Test
public void testDownloadFileFromBucket() throws UnsupportedEncodingException {
StringInputStream objectContent = new StringInputStream( "Lorem ipsum dolor sit amet.");
S3Object s3Object = new S3Object();
s3Object.setObjectContent(objectContent); Arrange inputs and mocks
when(this.amazonS3Client
.getObject( or(isA(String. class), isNull()), or(isA(String. class), isNull())))
Act: run method under test
.thenReturn(s3Object);
S3Object actualS3Object = this.cloudStorageService.downloadFileFromBucket( "bucket-name", "key")
assertSame(s3Object, actualS3Object);
}
 Desirable properties:
What is a unit test? •
•
Runs fast (a few ms)
Has no side effects on
other tests

@MockBean
private AmazonS3Client amazonS3Client;
@Test
public void testDownloadFileFromBucket() throws UnsupportedEncodingException {
StringInputStream objectContent = new StringInputStream( "Lorem ipsum dolor sit amet.");
S3Object s3Object = new S3Object();
s3Object.setObjectContent(objectContent); Arrange inputs and mocks
when(this.amazonS3Client
.getObject( or(isA(String. class), isNull()), or(isA(String. class), isNull())))
Act: run method under test
.thenReturn(s3Object);
S3Object actualS3Object = this.cloudStorageService.downloadFileFromBucket( "bucket-name", "key")
assertSame(s3Object, actualS3Object); Assert effects
}
Use case:
Refactoring
 1. git checkout a-test-refactoring

Refactoring 2. Right-click on PetTypeFormatter > Write tests

@Component
public class PetTypeFormatter implements Formatter<PetType> {
@Autowired
private PetRepository pets;
...
@Override
public PetType parse(String text, Locale locale) throws ParseException {

Collection<PetType> findPetTypes = this.pets.findPetTypes();

for (PetType type : findPetTypes) {
if (type.getName().equals(text)) {
return type;
}
}
throw new ParseException("type not found: " + text, 0);
}
}
 1. git checkout a-test-refactoring

Refactoring 2. Right-click on PetTypeFormatter > Write tests

3. Run tests with coverage
@ContextConfiguration(classes = {PetTypeFormatter.class, PetRepository.class})
@ExtendWith(SpringExtension.class)
public class PetTypeFormatterTest {
@MockBean(name = "petRepository")
private PetRepository petRepository;
@Autowired
private PetTypeFormatter petTypeFormatter;
…
@Test
public void testParse2() throws ParseException {
PetType petType = new PetType();
petType.setId(1);
petType.setName("Dog");
ArrayList<PetType> petTypeList = new ArrayList<PetType>();
petTypeList.add(petType);
when(this.petRepository.findPetTypes()).thenReturn(petTypeList);
assertSame(petType, this.petTypeFormatter.parse("Dog", new Locale("en")));
}
…
}
 1. git checkout a-test-refactoring

Refactoring 2. Right-click on PetTypeFormatter > Write tests

3. Run tests with coverage
4. Refactor PetTypeFormatter.parse
5. Run tests (a test fails)
@Component
6. Debug failing test
public class PetTypeFormatter implements Formatter<PetType> {
7. Fix bug
@Autowired
private PetRepository pets;
8. Run tests (all tests pass)
...
@Override
public PetType parse(String text, Locale locale) throws ParseException {

return this.pets.findPetTypes().stream()
.filter(type -> type.getId().equals(text))
.findFirst()
.orElseThrow(() -> new ParseException("type not found: " + text, 0));
}

}
Refactoring introduces
regression.
 Refactoring

Existing Diffblue Changed pass

Unit Run Commit
source Cover: source
tests Refactoring unit source code
code Create code tests and tests
unit tests

fail

Fix bug

● Improve test coverage

● Increase likelihood of catching
regressions introduced by refactoring
Use case:
New Feature
 1. git checkout a-test-new-feature

New Feature 2. Implement countDogs in PetTypeFormatter

3. Right-click on countDogs > Write tests

class OwnerController {
…
/**
* Counts the number of dogs belonging to the given owner.
* @param owner The owner
* @return The number of pets of type "Dog" belonging to the given owner
*/
public long countDogs(Owner owner) {
return owner.getPets().stream()
"Dog"))
.filter(pet -> pet.getName().equals(
.collect(Collectors.counting());
}
}
 1. git checkout a-test-new-feature

New Feature 2. Implement countDogs in OwnerController

3. Right-click on countDogs > Write tests
@Test 4. Review the created test
public void testCountDogs3() {
Pet pet = new Pet();
LocalDate birthDate = LocalDate.
ofEpochDay(1L);
pet.setBirthDate(birthDate);

Owner owner = new Owner();

owner.setLastName("Doe");
owner.setId(1);
owner.setCity("Oxford");
owner.setAddress("42 Main St");
owner.setFirstName("Jane");
owner.setTelephone("4105551212");
pet.setOwner(owner);

pet.setId(1);
pet.setName("Bella");

PetType petType = new PetType();

petType.setId(1);
petType.setName("Dog"); Unintended
pet.setType(petType);
behavior
HashSet<Pet> petSet = new HashSet<Pet>();
petSet.add(pet);
owner.setPetsInternal(petSet);

assertEquals(0L, this.ownerController.countDogs(owner));
}
 1. git checkout a-test-new-feature

New Feature 2. Implement countDogs in OwnerController

3. Right-click on countDogs > Write tests
@Test 4. Review the created tests
public void testCountDogs3() {
Pet pet = new Pet(); 5. Fix tests
LocalDate birthDate = LocalDate.
pet.setBirthDate(birthDate);
6. Debug and fix implementation
ofEpochDay(1L);

7. Run tests (pass)

Owner owner = new Owner();
owner.setLastName("Doe");
owner.setId(1);
owner.setCity("Oxford");
owner.setAddress("42 Main St");
owner.setFirstName("Jane");
owner.setTelephone("4105551212");
pet.setOwner(owner);

pet.setId(1);
pet.setName("Bella");

PetType petType = new PetType();

petType.setId(1);
petType.setName("Dog");
pet.setType(petType);

HashSet<Pet> petSet = new HashSet<Pet>();

petSet.add(pet);
owner.setPetsInternal(petSet);

assertEquals(1L, this.ownerController.countDogs(owner));
}
 ● Speed up test writing
● Demonstrate unintended behavior in
New Feature the implementation

Existing Changed Diffblue Candidate approved Commit

source Implement source Cover: unit Review
source code
code new feature code Create tests tests
and tests
Unit tests

unintended
behavior

Fix unit tests Correct Changed pass

Run
to match unit Fix bug in source unit
intended tests implementation code tests
behavior
fail
Use case:
Use in TDD
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

/**
* @GetMapping("/owners") public String processFindForm(Owner owner, ...)
* Look up the owner in the database by the given last name.
* If a single owner is found, redirect to /owners/{ownerId}.
* If several owners are found, allow selection of an owner in owners/ownersList.
*/

@WebMvcTest(controllers = {OwnerController.class})
@ExtendWith(SpringExtension.class)
public class OwnerControllerTest {

@Autowired
private MockMvc mockMvc;

@MockBean(name = "ownerRepository")
private OwnerRepository ownerRepository;

@MockBean(name = "visitRepository")
private VisitRepository visitRepository;
...
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

3. Run unit tests (fail)

…
@Test
public void testProcessFindForm_singleOwner() throws Exception {
Owner owner = new Owner();
owner.setLastName("Doe");
owner.setId(1);
owner.setCity("Oxford");
owner.setAddress("42 Main St");
owner.setFirstName("Jane");
owner.setTelephone("4105551212");
when(this.ownerRepository.findByLastName(or(isA(String.class), isNull())))
.thenReturn(Collections.singletonList(owner));

this.mockMvc.perform(get("/owners").param("lastName", "Doe"))
.andExpect(status().is3xxRedirection())
.andExpect(view().name("redirect:/owners/1"));
}
…
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

3. Run unit tests (fail)
…
@Test
public void testProcessFindForm_severalOwners() throws Exception {
Owner owner1 = new Owner();
owner1.setLastName( "Doe");
owner1.setId( 1);
owner1.setCity( "Oxford");
owner1.setAddress( "42 Main St");
owner1.setFirstName( "Jane");
owner1.setTelephone( "4105551212");

Owner owner2 = new Owner();

owner2.setLastName( "Doe");
owner2.setId( 1);
owner2.setCity( "Oxford");
owner2.setAddress( "1 High St");
owner2.setFirstName( "John");
owner2.setTelephone( "5121241055");

when(this.ownerRepository.findByLastName( or(isA(String. class), isNull())))

.thenReturn(Arrays. asList(owner1, owner2));

this.mockMvc.perform( get("/owners").param( "lastName", "Doe"))

.andExpect( status().isOk())
.andExpect( view().name( "owners/ownersList"));
}
…
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

3. Run unit tests (fail)
4. Implement processFindForm
public class OwnerController { 5. Run unit tests (pass)
… 6. Right-click on processFindForm > Write tests

@GetMapping("/owners")
public String processFindForm(
Owner owner, BindingResult result, Map<String, Object> model) {

Collection<Owner> results = this.owners.findByLastName(owner.getLastName());

if (results.size() == 1) {
owner = results.iterator().next();
return "redirect:/owners/" + owner.getId();
} else {
model.put("selections", results);
return "owners/ownersList";
}
}
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

3. Run unit tests (fail)
4. Implement processFindForm
public class OwnerControllerTest5.{Run unit tests (pass)
6. Right-click on processFindForm > Write tests
… 7. Review created tests
@Test
public void testProcessFindForm() throws Exception {

when(this.ownerRepository.findByLastName(or(isA(String.class), isNull())))
.thenReturn(new ArrayList<Owner>());

this.mockMvc.perform(get("/owners")) What’s the intended

.andExpect(status().isOk()); behavior in this case?
.andExpect(view().name("owners/ownersList"));
}
 1. git checkout a-test-tdd

TDD 2. Implement unit tests for OwnerController.processFindForm

3. Run unit tests (fail)
4. Implement processFindForm
public class OwnerControllerTest5.{Run unit tests (pass)
6. Right-click on processFindForm > Write tests
… 7. Review created tests
8. Fix unintended behavior in created tests
@Test
public void testProcessFindForm_notFound() throws Exception {

when(this.ownerRepository.findByLastName(or(isA(String.class), isNull())))
.thenReturn(new ArrayList<Owner>());

this.mockMvc.perform(get("/owners"))
.andExpect(status().isOk());
.andExpect(view().name("owners/findOwners"));
}
 1. git checkout a-test-tdd
2. Implement unit tests for OwnerController.processFindForm
TDD 3. Run unit tests (fail)
4. Implement processFindForm
5. Run unit tests (pass)
public class OwnerController { 6. Right-click on processFindForm > Write tests
… 7. Review created tests
8. Fix unintended behavior in created tests
@GetMapping("/owners")
public String processFindForm( 9. Amend implementation of processFindForm
Owner owner, BindingResult result, Map<String, Object> model) {

Collection<Owner> results = this.owners.findByLastName(owner.getLastName());

if (results.isEmpty()) {
result.rejectValue("lastName", "notFound", "not found");
return "owners/findOwners";
} else if (results.size() == 1) {
owner = results.iterator().next();
return "redirect:/owners/" + owner.getId();
} else {
model.put("selections", results);
return "owners/ownersList";
}
}
 1. git checkout a-test-tdd
2. Implement unit tests for OwnerController.processFindForm
TDD 3. Run unit tests (fail)
4. Implement processFindForm
5. Run unit tests (pass)
public class OwnerControllerTest6.{Right-click on processFindForm > Write tests
7. Review created tests
… 8. Fix unintended behavior in created tests
9. Amend implementation of processFindForm
@Test
public void testProcessFindForm1 throws Exception {
Owner owner = new Owner();
owner.setLastName("Doe");
owner.setId(1);
owner.setCity("Oxford");
owner.setAddress("42 Main St");
owner.setFirstName("Jane");
owner.setTelephone("4105551212");
when(this.ownerRepository.findByLastName(or(isA(String.class), isNull())))
.thenReturn(Collections.singletonList(owner)); What’s the intended behavior without
“lastName” parameter?
this.mockMvc.perform(get("/owners"))
.andExpect(status().is3xxRedirection())
.andExpect(view().name("redirect:/owners/1"));
}
 1. git checkout a-test-tdd
2. Implement unit tests for OwnerController.processFindForm
TDD 3. Run unit tests (fail)
4. Implement processFindForm
5. Run unit tests (pass)
public class OwnerController { 6. Right-click on processFindForm > Write tests
… 7. Review created tests
8. Fix unintended behavior in created tests
@GetMapping("/owners")
public String processFindForm( 9. Amend implementation of processFindForm
Owner owner, BindingResult result, Map<String, Object> model) {
// allow parameterless GET request for /owners to return all records
if (owner.getLastName() == null) {
owner.setLastName(""); // empty string signifies broadest possible search
}
Collection<Owner> results = this.owners.findByLastName(owner.getLastName());
if (results.isEmpty()) {
result.rejectValue("lastName", "notFound", "not found");
return "owners/findOwners";
} else if (results.size() == 1) {
owner = results.iterator().next();
return "redirect:/owners/" + owner.getId();
} else {
model.put("selections", results);
return "owners/ownersList";
}
Test-Driven Development
Existing Analyze Changed
Run fail Commit
source requirements Unit tests Implement source
unit source code
code & implement feature code
tests and tests
unit tests

pass

Amend requirements
& fix unit tests to Fix imple-
unintended match intended mentation
behavior behavior

Diffblue Additional Review approved

Cover: unit unit
Create tests tests
Unit tests
● Demonstrate gaps in requirements
● Speed up test writing
What’s more?
Diffblue Cover IntelliJ Plugin
• Community Edition: for open source only
• Paid plugin: for commercial software

Diffblue Cover CLI

• For integration into CI
• Try on https://www.diffblue.com/try-cover-browser
Takeaways
• Diffblue Cover is a tool that creates unit tests.
• Does not replace software developers
• Complements and speeds up manual test writing
• Created tests tell you what the current behavior
• Use cases:
• Refactoring: demonstrate unintended changes
• New feature: review demonstrates buggy implementation
• TDD: review demonstrates gaps in requirements
Giving Feedback
shorturl.at/auHLM
Please fill me in!

Getting Support
https://forum.diffblue.com
I’d like to have
this feature.
Cool stuff!
I don’t get tests!