Deploy without Fear:

Visual Regression
Testing with
Backstop.js
is here!

[email protected]
[email protected]
Ryan Bateman
Drupal Developer - Hook 42
@_porkloin_
porkloin
[email protected]
ryanbateman.space

[email protected]
Summary of Topics
1. What is Visual Regression Testing, and why should you
care?
2. What is Backstop, and how do I use it?
3. Backstop generator module!
4. Alternative software

[email protected]
 Syntax
Is your CSS malformed?

Project
4 Types of Did you meet internal code standards?

Reference
CSS Testing Do your styles render as expected?

Regression
After CSS changes are made, does the
page look as you expect it to?

[email protected]
 What is Visual
Regression
Testing?
And why should we care?

[email protected]
[email protected]
 Visual is a category of testing
software that focuses on
Regression identifying visual changes
between iterations or
Testing versions of a website.

[email protected]
 If you’re familiar with
the idea of a diff..

[email protected]
 Then you can understand visual regression
testing as a system for testing visual diffs.

[email protected]
[email protected]
Because we already
do this, but we do we
it poorly and slowly.

[email protected]
 Can you spot the difference?

[email protected]
Change Blindness:
a perceptual phenomenon that
occurs when a change in a visual
stimulus is introduced and the
observer does not notice it.

[email protected]
 Breaking CSS is easy,
testing it is hard.
-Garris Shippon,
BackstopJS creator

[email protected]
Manual Testing
is not only slow, but also
inaccurate.

[email protected]
 Or, as a project manager would say:

[email protected]
Let’s have
computers
do it for us!

[email protected]
 What is

BackstopJS
and how can we use it?

[email protected]
[email protected]
 BackstopJS is a
framework for conducting
visual regression tests
written in NodeJS. It
creates visual diffs and
provides easy-to-configure
test parameters for a
variety of viewport sizes
and pass/fail conditions.
[email protected]
 ● Renders screenshots in a headless
test environment using Chrome
Headless, Phantom, or Slimer.
How does ● Simulates user interactions with
Puppeteer, ChromyJS and
it work? CasperJS scripts
● Generates visual diffs using
Resemble.js

[email protected]
Why Backstop?
1. Ease of use and
configuration.
2. Reliability (compared
to older headless testing
tools).
3. Integration with JS task
runners and CI systems.

[email protected]
Installation & Configuration
It’s really simple!

Installation (global): npm install -g backstopjs

Installation (local): npm install --save-dev

backstopjs

Configuration: backstop init (creates backstop.json template)

[email protected]
Anatomy of backstop.json (part 1)
"id": "my_visual_regression_test",
"viewports": [
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
}
],
"onBeforeScript": "chromy/onBefore.js",
"onReadyScript": "chromy/onReady.js",

[email protected]
Anatomy of backstop.json (part 2)
"scenarios": [
{
"label": "My Great Homepage", "postInteractionWait": "",
"cookiePath": "selectors": [],
"backstop_data/engine_scripts/cookies.json", "selectorExpansion": true,
"url": "https://mygreat.site/", "misMatchThreshold" : 0.1,
"referenceUrl": "", "requireSameDimensions": true
"readyEvent": "", },
"readySelector": "", …
"delay": 0, ...
"hideSelectors": [], ],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",

[email protected]
Anatomy of backstop.json (part 3)
"paths": {
"bitmaps_reference": "backstop_data/bitmaps_reference",
"bitmaps_test": "backstop_data/bitmaps_test",
"engine_scripts": "backstop_data/engine_scripts",
"html_report": "backstop_data/html_report",
"ci_report": "backstop_data/ci_report"
},
"report": ["browser"],
"engine": "chrome",
"engineFlags": [],
"asyncCaptureLimit": 5,
"asyncCompareLimit": 50,
"debug": false,
"debugWindow": false

[email protected]
Scenarios, in-depth
● A Scenario isn’t necessarily a page – we may create
several scenarios for one single page.
● Scenarios reflect the state of the UI given a
predetermined set of scenario properties.
● Scenario properties can include interaction (clicking,
hovering, etc.), timing (delay, on load of a selector, etc.),
or even client-side data (cookies).

[email protected]
 backstop reference
Create a series of baseline/reference images for
Great, but future tests to be conducted against.

how do I backstop test

use Take a new series of sceenshots and compare
them to the reference set, then open a report.
Backstop?
backstop accept
Use the last set of screenshots as the new
reference, accepting them as the new canonical
[email protected] version to test against
Let’s see it in action!

Obligatory live demo warning:

this might go down in flames.

[email protected]
Backstop’s ease of configuration makes it great as a kind of
personal QA for front-end devs.

[email protected]
Scenario: Backstop as Personal QA
● We’ll assume we’re a junior front-end developer
named Jean. Jean works at a small web agency.
● We have a simple CSS fix for a new maintenance
client.
● The site doesn’t have any front-end tooling set up, and
certainly not any visual regression testing. Yikes!
● Jean’s PM wants this done yesterday.

[email protected]
What does Jean do?
● Before making any styling changes, Jean installs backstop globally (npm
install -g backstopjs) creates a new directory to house our test,
called projname-backstop
● In a terminal from projname-backstop, Jean executes backstop init.
● Jean modifies the generated backstop.json scenarios with a
representative sampling of pages across the website.
● From proj-backstop, Jean executes backstop reference to create a
baseline set of images.
● After completing work, Jean runs backstop test, reviews the results to
ensure the changes seen in the screens are as intended, and commits the
styling changes as usual!
[email protected]
Integrated into an existing front-end toolchain, Backstop can
help extend existing test coverage to include visual
regression.

[email protected]
Scenario: Backstop in existing front-end tools

● We’ll assume we are a senior front-end developer

named Casey.
● Casey is building out a custom Drupal subtheme for a
client who wants a fresh redesign.
● Never one to be a slouch, Casey wants to make sure the
new theme is architected with test-driven development in
mind!

[email protected]
Methods of Integration with FE Tools
● Once installed as a local package to our project, backstop can be used as
an npm script in our package.json file:
○ "scripts": {
"approve": "backstop approve",
"test": "backstop test",
"init": "backstop init"
}
● In JS taskrunners like gulp, we can import backstop and call backstop
functions:
○ const backstopjs = require('backstopjs');

gulp.task('backstop_reference', () => backstopjs('reference'));

gulp.task('backstop_test', () => backstopjs('test'));

[email protected]
 ● The backstop_data directory
contains all of the screenshots that
What about are generated from testing, and its
contents should be listed in
source .gitignore
● backstop.json should be
control? committed, and in the case of
front-end toolchains, backstop tasks
should be added to any aggregated
test/pre-deploy tasks
[email protected]
Great, but
defining It is tedious!
scenarios But that’s okay, because
sounds there’s a module for that.
tedious.

[email protected]
 Backstop
Generator
a backstop.json generator for Drupal 8

[email protected]
[email protected]
 www.drupal.org/project/backstop_generator

[email protected]
 Backstop Generator is a
What is Drupal 8 module that
backstop creates backstop.json
configuration files based
generator? on the site’s unique
content.

[email protected]
 Backstop generator can definitely do
more! If you use it and want to
contribute to building more features,
join the issue queue on drupal.org,
submit a patch, or report a bug!

[email protected]
 Questions?

[email protected]
 Thank you!

[email protected]