Get Started with Visual Regression Testing

and WebdriverIO
Kevin Lamping and Micah Godbolt
This book is for sale at http://leanpub.com/visual-regression-testing-and-webdriverio-guide

This version was published on 2017-02-21

This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing
process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and
many iterations to get reader feedback, pivot until you have the right book and build traction once
you do.

© 2016 - 2017 Kevin Lamping and Micah Godbolt

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

What to know about Visual Regression Testing . . . . . . . . . . . . . . . . . . . . . . . . 2

Some Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
The Benefits of Functional Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Some Disclaimers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
The Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Course Requirements (aka today’s homework) . . . . . . . . . . . . . . . . . . . . . . . . 4
What’s Next? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Today you’re going to write your first regression test . . . . . . . . . . . . . . . . . . . . 5

WebdriverIO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
WebdriverCSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

ChromeDriver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Let’s make a test! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Real Visual Regression Testing with WebdriverCSS . . . . . . . . . . . . . . . . . . . . . . 9

GraphicsMagick . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Setting Up WebdriverCSS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Writing our first Visual Regression Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Hey, that wasn’t a test! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Seeing what’s wrong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
I meant to change that… . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Up next . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

And… Action! Take Control with WebdriverIO Commands . . . . . . . . . . . . . . . . . 13

Starting Off . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Clicking around . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Finishing up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
The Full Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
In Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Advanced WebdriverCSS Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

CONTENTS

Customize the setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Customize a screenshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Excluding, hiding and removing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Managing Baseline images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Updating baseline images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

What’s Next for You? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

That’s It . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Preface
The content for this book is from learn.visualregressiontesting.com1 . It is a 6-day email course
available for anyone interested in Visual Regression Testing.
Throughout this book, you’ll notice mentions of the current day in the course (e.g. “It’s Day 1!”), or
references to previous/later days. This is due to the nature of the original content.
Each chapter in this book matches a day in the course. So six chapters is equal to six days.
With that said, let’s get started!
1
http://learn.visualregressiontesting.com
What to know about Visual
Regression Testing
Hey Folks, Micah and Kevin here from visualregressiontesting.com.
Over the next week, you’re going to learn the essentials Visual Regression Testing.
You may be surprised by how much fun writing these tests will be. After all, you’re teaching a
computer how to use your website! That’s pretty neat in our book.

Some Terminology
Let’s kick it off by expanding our vocabulary. We’ll be using these terms throughout the course:

Regression Testing
Change hurts. Especially when it’s unwanted.
“Regressions” are changes, bad ones, to the functionality of your site. To say you’ve found a
“regression” means you found code that used to work but no longer does.
“Regression Testing” is a type of testing that checks for these bugs in existing functionality after an
update.
While you definitely want to put new code through the wringer, it’s also important to look at existing
features to ensure they weren’t adversely affected by the updates. That’s what regression testing
does.

Automated Testing
Automation comes in many forms, whether through robots or driverless cars. For us, it’s writing
code that “automates” actions on a website, similar to how AI is written to “automate” driving a car.
While the goal of website automation is to test without a human clicking links and entering text, in
no way can it take the place of hard-working people.
First of all, someone has to write the automation and know how to keep it up to date. That can easily
be a full-time job on a large enough application.
But there is also a lot of nuance to websites. Checking to see if an animation worked as designed is
very difficult with automated tests. Automation simply aims to handle the boring, repetitive tasks,
leaving you time to test the hard stuff.
What to know about Visual Regression Testing 3

Functional Testing
There are many types of software testing out there; Wikipedia has an extensive list of the various
definitions2 .
We’re focusing on “functional” testing, which checks the UI functionality of a website and ensures
that it works properly. This is not the same as testing JavaScript functions (that would be unit testing).
Functional testing also goes by “system testing”, “end-to-end tests” and many other monikers, but
for our course we’ll use the term “functional testing”.

Visual Testing
There’s one last term to cover, and that’s the concept of “visual testing.” In this course, we’ll talk
about both standard testing (e.g. make sure certain text is on the page) and visual testing (e.g. make
sure the page looks the same as before).
Visual Regression Testing has a unique set of benefits. Just because an element is on a page does not
mean it’s in the right location (or the right size or the right color).
With visual testing, we take screenshots of various parts of the page to serve as the “baseline” of
how the site should work. Then in subsequent test runs, we take new screenshots and compare to
those baselines. Any differences are flagged for review.

The Benefits of Functional Testing

Why take the time to learn all of this? Is it really worth it to write code that only tests the code you
really should be writing?
Here’s the thing: you’re already doing this. Every time you refresh your page to see a change you’ve
made, you’re running a manual regression test. Throughout the lifetime of a website, hundreds of
hours are spent regression testing it.
But the truth is, we’re bad at this. We either get lazy and skip a few tests, or we unknowingly miss
a defect or two (we’re especially bad at detecting small visual changes3 ).
With automation, regression testing the site becomes less of a repetitious chore, so we’re more likely
to do it on a consistent basis. You can even set up hooks to automatically test your code after every
change.
And computers are much better at spotting pixel changes. It’s all numbers to the computer, and
numbers are easy to compare.
2
https://en.wikipedia.org/wiki/Software_testing#Testing_types
3
https://en.wikipedia.org/wiki/Change_blindness
What to know about Visual Regression Testing 4

Some Disclaimers
We really believe in the benefits of automated testing, so we should probably throw out some
disclaimers before we get you too excited:

• Functional testing can be finicky, because websites are complicated.

• You can’t test everything because, again, websites are complicated.
• Automation doesn’t work in all browsers, because some browsers just don’t support it. You’re
best bets are going to be Firefox, Chrome and Internet Explorer.

Basically, it’s not magic (although it feels like it at times). There are certain things you just won’t be
able to automate testing for, so it’s best not to fight that fact.

The Technology
We’ll go in to installation of the tools tomorrow, but here’s what we’ll be using:

• Selenium4 - A browser automation tool

• WebdriverIO5 - A Node.js library to talk to Selenium
• WebdriverCSS6 - An add-on to WebdriverIO to help with Visual Regression Testing

Course Requirements (aka today’s homework)

There are a few requirements you’ll need to have going before getting started:

• A working NodeJS environment7 (Learn about Node and NPM if you are new to it8 )
• A light understanding of the command line9
• An understanding of JavaScript fundamentals10

What’s Next?
Phew, you’ve made it through the entire first day without any code being shown. Don’t worry, we’ve
saved all that for tomorrow, where we’ll take a look at writing your very first automated test (we’re
so excited for you!)
4
https://en.wikipedia.org/wiki/Selenium_(software)
5
http://webdriver.io/
6
https://github.com/visualregressiontesting/webdrivercss
7
https://github.com/creationix/nvm#node-version-manager-
8
https://docs.npmjs.com/getting-started/what-is-npm
9
https://www.codecademy.com/learn/learn-the-command-line
10
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Grammar_and_types#Basics
Today you’re going to write your first
regression test
Day 2 is here; we hope you’re ready to dig in. Let’s start with some installs!
First, create a folder to store all of your code. You can name it whatever you want.
Then, open a command line and navigate to that folder. From there, you’ll create a new package.json
file by running the command:

npm init -y

If you’re new to NodeJS or unfamiliar with NPM, have a look at their about page for a quick intro
video11 .
This file just stores information about your project and the software you need. It’s useful to have on
hand if you need to set the project up again later.
Now to run the actual installation:

npm install --save webdriverio visualregressiontesting/webdrivercss chromedriver

WebdriverIO
WebdriverIO is a JavaScript functional test library. It lets you write test instructions in JavaScript,
then passes them along to Selenium, which tells the browser what actions to take.
For us, writing tests in JavaScript is awesome. We don’t have to learn Java for test automation!

WebdriverCSS
WebdriverCSS is a Visual Regression Testing plugin for WebdriverIO. It’s a pretty basic tool that
boils down to two basic functions:

1. Capture images of the specified portion of the website.

2. Compare those images to previous ones, creating a “diff” version if differences are found.

You can do a fair amount of configuration with the tool, but for now, those two concepts are all you
need to know.
11
https://docs.npmjs.com/getting-started/what-is-npm
ChromeDriver
We’re testing in real browsers folks, which is really quite awesome. But it does require some set up.
While getting Selenium running is easier than ever, it’s too complex for a single email.
As a worthy alternative, we’re going to use ChromeDriver. ChromeDriver is a Selenium-like tool
that allows us to run tests on a real Chrome browser.
To get it going, just open a new command line window (making sure you’re in your project folder),
and run:

./node_modules/.bin/chromedriver --url-base=/wd/hub --port=4444

This will be a constantly-running service. You can stop the it by pressing Ctrl-C.

• If you’re interested in why we reference ./node_modules/bin/ and a way around that, read
up on npm scripts12 .

Let’s make a test!

Now that we have ChromeDriver running, we can use it to run some tests. In your project folder,
create a new file called tests.js. Open it up in your favorite code editor.
The first thing we’ll do is load WebdriverIO in to the file. To do that, we’ll use a require statement:

var wdio = require("webdriverio");

Disclaimer For the rest of this course we’re going to leave out repetitive bits of the code
in our examples. To see the full code, check out the code samples for each day13 .

With that loaded, our next step is to set up a browser instance.

12
http://firstdoit.com/npm-scripts/
13
http://learn.visualregressiontesting.com/code-samples.zip
ChromeDriver 7

var browser = wdio.remote({

desiredCapabilities: {
browserName: "chrome"
}
}).init();

What just happened here? Well, we told WebdriverIO to start a new Chrome browser. If you want
to read more about it, check out the documentation14 .

Get to our page

Now that we have a browser to play around with, let’s do some damage. We’re going to request a
website by using the url browser command.

browser.url("http://learn.visualregressiontesting.com");

Check the title

Now that we have a page loaded, we can do stuff with it. Let’s validate it’s actually the right place.
How about checking the title of the page and logging it through console.log? Here’s what the
syntax looks like:

browser.url("http://learn.visualregressiontesting.com")
.getTitle().then(function(title) {
console.log("Title is: " + title);
});

Here we asked our browser to get the title of the page (the aptly named getTitle command), and
then logged that value out using the generic then command.
Note: If you’re not familiar with JavaScript Promises, this syntax may look a little strange. For the
sake of brevity, we’re not going to get into the details of it in this email. Fortunately there are a lot
of resources on JavaScript Promises15 already out there for you to find.

End the test

Finally, we need to tell WebdriverIO to shut things down. That’s as simple as adding .end() at the
bottom of our commands:
14
http://webdriver.io/guide/getstarted/configuration.html
15
http://lmgtfy.com/?q=javascript+promise+tutorials
ChromeDriver 8

browser.url("http://learn.visualregressiontesting.com")
... tests are here ...
.end();

Just to recap, here’s what your tests.js file should look like:

var wdio = require("webdriverio");

var browser = wdio.remote({
desiredCapabilities: {
browserName: "chrome"
}
}).init();

browser.url("http://learn.visualregressiontesting.com")
.getTitle().then(function(title) {
console.log("Title is: " + title);
})
.end();

Now it’s time to run the test! You can do so by asking Node (via the command line) to ever so kindly
execute your code:

node tests.js

If all went well, you should have seen the following message in your command line:

Title is: Learn Visual Regression Testing

Tomorrow’s outlook
So, how does it feel to have programmatically taken control of a browser? A little awesome, right?
Okay, if you’re underwhelmed, that’s because we didn’t really test much. We only peeked at the
page title then closed everything down before we got ourselves in to real trouble.
We promise that over the next couple of days we’ll get our hands extra dirty with mouse clicks,
keyboard taps, HTML checks and very real visual tests.
If you can’t wait until then, take a look at the WebdriverIO API page16 to see a full list of commands
available for use.
16
http://webdriver.io/api.html
Real Visual Regression Testing with
WebdriverCSS
It’s Day 3! We’ve spent the past two days getting things in place. Today is the day we finally use
WebdriverCSS!

GraphicsMagick
So we need to install one more tool.
GraphicsMagick is software that performs image manipulation and comparison. It allows us to
compare our images, and let us know when there are differences between the two.
To install it, follow the instructions in the WebdriverCSS Readme17 . Be forewarned that you may
need to uninstall ‘ImageMagick’ if you have it installed. Yes, they’re different software, and they
don’t always get along.
To test your installation, run gm from the command line. You should see a response that looks like:

GraphicsMagick 1.3.21 2015-02-28 Q8 http://www.GraphicsMagick.org/

Copyright (C) 2002-2014 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.
Usage: gm command [options ...]

Setting Up WebdriverCSS
Now that we have that installed, it’s time to make some changes to our tests.js file. First, we load
and initialize WebdriverCSS.
On a new line after your ‘.init(…)’ code, add:

require('webdrivercss').init(browser);

This appends the ‘webdrivercss’ command to our browser object.

Writing our first Visual Regression Tests

Let’s replace that almost useless ‘getTitle’ command with a much more useful ‘webdrivercss’ one:
17
https://github.com/visualregressiontesting/webdrivercss#install
Real Visual Regression Testing with WebdriverCSS 10

browser.url("https://learn.visualregressiontesting.com")
.webdrivercss("homepage", [
{
name: "header",
elem: ".header"
},
{
name: "benefits",
elem: ".benefits",
screenWidth: [320,640,1024]
}
])
.end();

In our code, we’re capturing two different elements on our homepage: the header and the benefits
section.
The header doesn’t change over different breakpoints, but the benefits section does. So to make sure
we test each of those breakpoints we use the screenWidth parameter in the second set of options.
This will produce a total of three images for the benefits element, one for each viewport size.
If you’d like to learn more about the options available to you, check out the documentation18 .

Is your ChromeDriver Running?

Remember last email where we had you start ChromeDriver? Yeah, do that again:

./node_modules/.bin/chromedriver --url-base=/wd/hub --port=4444

Run Our Tests

If you need to catch up, or check your work, take a look at day-3.js in the course code
samples19 .

With ChromeDriver started, and WebdriverCSS ready to go, let’s run our tests!

node tests.js

To see the results of our tests, open up the newly created webdrivercss folder. In this folder you’ll
see the following files:
18
https://github.com/visualregressiontesting/webdrivercss#usage
19
http://learn.visualregressiontesting.com/code-samples.zip
Real Visual Regression Testing with WebdriverCSS 11

homepage.320px.png
homepage.640px.png
homepage.1024px.png
homepage.png
homepage.benefits.320px.baseline.png
homepage.benefits.640px.baseline.png
homepage.benefits.1024px.baseline.png
homepage.header.baseline.png

The first four images listed (without baseline in their name) are the full page screenshots
WebdriverCSS captures from Chrome. Once those images are captured, WebdriverCSS crops the
image down to your specified areas giving you the four baseline images.

Hey, that wasn’t a test!

So you may have noticed that we didn’t actually test anything. We only took some screenshots of
a working page. It isn’t until you run your tests with baseline images already there that you’ll see a
possible diff.
Normally this happens after you’ve done some edits to your page and want to make sure it’s working
right. For use, we’re going to change the url of the page we testing, to simulate a “broken” update.
On the browser.url call, change the URL to include /broke.html:

browser.url("https://learn.visualregressiontesting.com/broke.html")

Seeing what’s wrong

While the test still ran normally, if you check your webdrivercss folder, you may notice two things:

1. There are now regression versions of the images

2. The diff folder now contains four colorfully pink images.

The regressions are the ‘next state’ version of your image. It’s useful to see the actual image that
was captured.
The ‘diff’ images are a comparison of the baseline and regression images, highlighting in hot pink
anything dissimilar between the two.

I meant to change that…

If you ever want to recreate your baselines, simply delete the webdrivercss folder, run your tests
again with your updates in place, and you’ll have a new set of baselines for use.
Real Visual Regression Testing with WebdriverCSS 12

Up next
Snapping pictures of the initial page load is good, but what about dynamic content? In our next
lesson, we’ll learn some advanced features of WebdriverIO and see how to test interactive elements
like mobile navigation and login forms.
And… Action! Take Control with
WebdriverIO Commands
Yesterday we learned to use WebdriverCSS for capturing screenshots of our page. Today, we’re going
to take things one step further. Scratch that. We’re going to things several steps further by adding
“steps” to our tests.
Websites aren’t static. When you visit one, you’re taking all sorts of actions. You’re clicking buttons,
entering text, dragging boxes around. There’s a lot to do!
So far though, our tests haven’t done much. We’ve simply loaded our page, checked a title, and taken
a few screenshots. What about dynamic elements like dropdown menus and form fields? We should
totally test those too!
For example, if we wanted to test the main navigation of the outdatedbrowser.com website, our steps
would look something like this:

1. Load the page

2. Screenshot the menu icon
3. Click the menu icon (opening the nav)
4. Screenshot the menu
5. Click a main nav link (New page loads automatically)
6. Screenshot the menu icon

Starting Off
We’re testing a new page, so we need a new file. Let’s fill it in with the basics:

var wdio = require("webdriverio");

var browser = wdio.remote({
desiredCapabilities: {
browserName: "chrome"
}
}).init();

require('webdrivercss').init(browser);

var menuIcon = {
And… Action! Take Control with WebdriverIO Commands 14

name: "Menu Icon",

elem: ".menu a"
};

browser.url("http://outdatedbrowser.com")
.webdrivercss("Hamburger Icon", [menuIcon])
.end();

Just a reminder to make sure your ‘chromedriver’ instance is running

So far there’s nothing new. We’ve moved our WebdriverCSS selector to its own variable, and the
reason why is coming up next!

Clicking around
The navigation on outdatedbrowser.com is hidden by default behind a friendly hamburger icon.
After having taken a picture of this icon using WebdriverCSS, we should click it to open the menu.
To do this, we’re going to use the ‘click’ command. And just like with the WebdriverCSS command,
we’ll pass in a CSS selector to specify the element we want to click:

browser.url("http://outdatedbrowser.com/en")
.webdrivercss("Main Menu Icon", menuIcon)
.click(menuIcon.elem)

See why we needed that menuIcon variable? Since we’re using the same selector for both Web-
driverCSS and our click command, it’s nice to keep it all in one variable.
FYI, click works almost exactly like an actual left click with your mouse. That’s the beauty of
Selenium; it’s as if you took the action yourself!

Capturing the opened menu

After clicking the menu icon, it takes a second to animate in. If we took the screenshot right away,
we’d get some strange results. We’re going to add another WebdriverCSS test to capture the opened
menu, but before that we need to use the pause command to wait a second while the menu opens:
And… Action! Take Control with WebdriverIO Commands 15

var menuIcon = {...};

var menu = {
name: "Menu",
elem: ".main_menu"
};

browser
... previous actions ...
.pause(1000)
.webdrivercss("Main Menu - Open", menu)

The pause command takes a number of milliseconds to wait. Since we want to wait a second, we’ll
pass in the number 1000, or 1000 milliseconds.
Now look at that! We have a Visual Regression Test for a dynamic element that animates in. That’s
complicated stuff!

Click a link
It would be good to know how the page looks after clicking a menu item and loading a new page.
We’ll use the ‘click’ command again to trigger a page load of the ‘Project’ menu item:

browser
... previous actions ...
.click("=THE PROJECT")

Notice anything different about this click command? In order to let WebdriverIO know which link
we want, we passed in an equal sign plus the text of the link.
Not only does WebdriverIO have strong support for CSS selectors, but it also supports text-based
selectors as well. Check out the selectors page20 for more information.

Finishing up
Last thing; Now that we’re on the Projects page, we should open the menu and capture the
‘active’ state of the project menu item. Even though we’ve loaded an entirely different webpage,
WebdriverIO is smart enough to understand what’s going on. We can write our test as one cohesive
script:

20
http://webdriver.io/guide/usage/selectors.html
And… Action! Take Control with WebdriverIO Commands 16

browser
... previous actions ...
.click("=THE PROJECT")
// WebdriverIO automatically waits here while the new page loads
.webdrivercss("Main Menu Icon - Projects", menuIcon)
.getUrl().then(function(url) {
console.log("Page url is: " + url);
})
end();

We threw the getUrl command in just to validate we’re on the right page, although our menu
screenshot should be validation enough.

The Full Tests

That sure was a lot, huh? Be sure to check out the code samples21 for Day 4 to see it all together.

In Review
Wow! We just pieced several parts together to make one very useful test. Hopefully you feel
more empowered in your testing ability. Check out the full WebdriverIO commands API22 to see
everything you now have the power to do.
Tomorrow, we’re looking at advanced WebdriverCSS usage. Until then, try writing some in-depth
tests on your own!
21
http://learn.visualregressiontesting.com/code-samples.zip
22
http://webdriver.io/api.html
Advanced WebdriverCSS Usage
As you advance in your test writing capabilities, you’ll find yourself in situations where a plain ol’
screenshot just won’t work. Maybe you need to test all images in three different resolutions, or you
need to hide some text because it’s always changing.
Good news, WebdriverCSS has you covered.

Customize the setup

When we first set up WebdriverCSS, we glossed over the fact that you can customize some global
options it has.
When initializing WebdriverCSS, our command usually looks like:

require('webdrivercss').init(browser);

There’s a second argument we can pass in to that ‘init’ call; an configuration object which defines
these aspects of our screenshots:

• screenshotRoot
– By default, screenshots are all saved to the ‘./webdrivercss’ folder. By passing in a custom
path, we can change where all the baseline screenshots are saved.
• failedComparisonsRoot
– Similar to screenshotRoot, this option defines where to store the ‘diff’ images. It defaults
to ./webdrivercss/diff.
• misMatchTolerance
– A number between 0 and 100 (default is 0.05) that defines the degree of mismatch to
consider two images as identical. Increase the number to be less strict in comparisons;
decrease to define smaller difference limits.
• screenWidth Numbers[] ( default: [] )
– When defined, will cause every capture to be taken in the different screen widths
requested
• updateBaseline
– Set this to true to automatically update the baseline images with the latest capture. Useful
for when you make an update to your design and want to redefine the images.

Here’s what these options would look like in regular use:

Advanced WebdriverCSS Usage 18

require('webdrivercss').init(browser, {
// example options
screenshotRoot: 'my-shots',
failedComparisonsRoot: 'diffs',
misMatchTolerance: 0.05,
screenWidth: [320,480,640,1024],
updateBaseline: true
});

Customize a screenshot
In our previous usage of the WebdriverCSS command, we showed example code but didn’t go on to
detail on the function itself. Let’s do that now!
The webdrivercss function takes 3 different parameters:

1. An ID: Each WebdriverCSS test needs to have a unique ID. This value will be the prefix for
all of the screenshot file names, and should therefore follow traditional naming conventions
like no spaces, dashes, or special characters.
2. Options: an array of option objects, each one representing a different part of the page that
you would like to test. Properties in the objects include:
3. name: Name of the captured element
4. elem: Selector of the element you want to capture
5. width: You can specify a fixed width for your screenshot
6. height: You can specify a fixed height for your screenshot
7. x: You can specify a fixed x coordinate for your screenshot (requires width/height)
8. y: You can specify a fixed y coordinate for your screenshot (requires width/height)
9. screenWidth: Pass through an array of screen widths to test this element at
10. Various other properties to hide, remove or exclude23 parts of the page, which can be useful
in order to ignore dynamic components such as advertisements. We’ll talk about these in a
moment.
11. The Callback: This function answers the “Ok I’ve got all of these images, what do I do now”?
We’ll leave it alone, but you can customize it gain information on the status of your screenshot
comparisons24 .

Here’s what a fully customized capture could look like:

23
https://github.com/webdriverio/webdrivercss/tree/beta-rc1#usage
24
https://github.com/webdriverio/webdrivercss#let-your-test-fail-when-screenshots-differ
Advanced WebdriverCSS Usage 19

webdrivercss('startpage',[
{
name: 'header',
elem: '#header'
}, {
name: 'hero',
elem: '#hero'
}, {
name: 'headerbar',
x: 110,
y: 15,
width: 980,
height: 34,
screenWidth: [1200]
}
])

You can peruse through all the options via the documentation25 .

Excluding, hiding and removing

We mentioned the ability to hide content, but didn’t show you an example. That’s because it deserves
its own shoutout.
Sometimes it is unavoidable that content constantly changes inside of a screenshot. This would break
all your tests, all the time.
To prevent this you can determine areas, which will either be excluded, hidden or removed.

• Excluded
– This will cause the area or element defined to be covered in black
• Hidden
– This will cause the element defined to have ‘visibility: hidden’ applied to it. It will still
take up space on the page, only being invisible.
• Removed
– Uses ‘display: none’ to remove the defined element from the page, as if the element
didn’t exist on the page at all.

Wanna see what that looks like? Of course you do!

25
https://github.com/webdriverio/webdrivercss#usage
Advanced WebdriverCSS Usage 20

.webdrivercss('header', {
name: 'header',
elem: '#header',
exclude: ['#leaderboard-ad', '#mini-ad'],
hide: '.username',
remove: '#date'
});

Managing Baseline images

This is the last thing we’ll talk about today.
If you’re using a version control system like Git, we recommend storing baseline images in it like
any other file (although you may want to use a .gitignore file26 to skip the .regression. and diff
images). This makes it simple to share baseline images across teams. It also allows for changes in
the baseline to be tracked over time and viewed in pull requests.
Be aware that if team members or testing platforms are not using the same OS, the baseline images
might differ slightly. A common example is that a baseline will work for local testing on OSX, but the
same baseline image fails when TravisCI runs the same visual regression test since it is Linux-based.

Updating baseline images

Eventually our design is going to change, and when it does, we need to make sure that our new
baseline images accompany our style changes.
If we increase the font size of our main header, our commit should include not just the new css, but
the new baseline file as well. The reason for this is so that the next person that downloads our new
CSS will also have a baseline image of our header with that larger text.
26
https://github.com/webdriverio/webdrivercss/blob/master/examples/.gitignore
What’s Next for You?
First off, congrats on making it to the final day! Your reward is an email with no code in it. Exciting,
right?
There are quite a few topics we’d like to briefly touch upon to get your gears turning on what to do
next. We’ll stray away from the specific technical implementations and talk general ideas.
Without further ado, here’s your Top 10 list of “Ways to Test Better”:

1. Know that you are not limited to a single browser when testing. Pass in an array of browsers27
in your desiredCapabilities configuration and WebdriverIO will run your tests in each
browser.
2. WebdriverIO has built in support28 for Selenium cloud tools like Sauce Labs29 and Browser-
stack30 . These services are great for testing in a multitude of browsers, and provide consistency
when snapping screenshots through WebdriverCSS.
3. Automation and Continuous Integration go hand in hand. Getting hooked in to a tool like
Jenkins31 or TravisCI is a great next step in improving the value of your tests.
4. Sometimes you just need to see how someone else did it. The WebdriverIO code repo contains
several examples to browse through32 .
5. Get your tests organized by using a test framework like Mocha33 or Jasmine34 . These
frameworks make working with large test suites a breeze, and offer a deep set of features.
6. Once you start building out several test files, you’ll end up with that same WebdriverIO
configuration scattered across all of them. Switch over to using the WebdriverIO Test Runner35
to simplify your test setup.
7. Get your team on board with Visual Regression Testing by bribing them with cookies and
donuts.
8. Need more help? The WebdriverIO36 and WebdriverCSS37 chat rooms are open at all hours
for questions and help with the journey. Reach out to @micahgodbolt and @klamping to let
us know you’ve joined the fun.
27
https://github.com/webdriverio/webdriverio/blob/master/examples/wdio.conf.js#L71
28
http://webdriver.io/guide/testrunner/cloudservices.html
29
https://saucelabs.com/
30
https://www.browserstack.com/
31
http://webdriver.io/guide/testrunner/jenkins.html
32
https://github.com/webdriverio/webdriverio/tree/master/examples
33
http://mochajs.org/
34
https://github.com/jasmine/jasmine
35
http://webdriver.io/guide/testrunner/gettingstarted.html
36
https://gitter.im/webdriverio/webdriverio
37
https://gitter.im/webdriverio/webdrivercss
What’s Next for You? 22

9. For more information on Visual Regression Testing, check out the main website at visualre-
gressiontesting.com38 .
10. Want to learn more WebdriverIO? Check out the full course on it for in-depth video lessons39 .

That’s It
Again, thanks for reading through these emails and kudos on finishing the course.

38
http://visualregressiontesting.com
39
http://learn.webdriver.io/