W H I T E P A P E R – M AY 2 0 2 0

The CIO’s Guide to CI/CD

By Dormain Drewitz and Brian Roche
The CIO’s Guide to CI/CD

Table of contents
Introduction 3
The agile trinity: Microservices, platforms and CI/CD 3
Fast feedback and tight focus 4
First: You must write the tests 5
Continuous integration explained 5
Continuous delivery explained 6
CI/CD in action 6
CI/CD as an organizational change agent 7
How CI/CD affects end users and customers 8
Modern software is all about agility 8
Additional reading 9
About Intersect 9

W H I T E PA P E R | 2
The CIO’s Guide to CI/CD

This guide was co-authored by Introduction

Dormain Drewitz of VMware and It’s common in talks about digital transformation to make fun of waterfall as a
Brian Roche of Cognizant. software delivery methodology, but that’s not entirely fair. Waterfall works great
if your requirements are unchanged, you understand your users and, as a result,
you don’t need to be highly adaptable to users’ changing needs.

As with most things these days, though, those stable, slow-moving conditions are
becoming less and less common. Today’s users expect fast bug fixes, new features
and regular updates. Similarly, today’s competitors are moving faster, nimbly adding
new service offerings and addictive features. And good developers, who should put an
emphasis on user experience, don’t want undue process getting in the way of necessary
progress. Perhaps that’s why, according to the 2018 Standish Group Chaos Study,
21 percent of waterfall software projects are considered failures, compared with only
8 percent of agile projects.1

However, going agile alone doesn’t ensure success. The same Standish Group study
also found that 50 percent of agile projects were challenged—only slightly lower than the
53 percent of waterfall projects that were challenged. This begs the question of why agile
projects get stuck so frequently? And, more importantly, how organizations can unstick
their agile projects.

The agile trinity: Microservices, platforms and CI/CD

Agile is only part of delivering software faster. While there’s no one-size-fits-all solution
to effective agile delivery, there are three common factors that successful enterprise IT
leaders often cite when discussing how they achieve high deployment velocity:

1. A shift to microservice architecture patterns

2. Implementation of platform-centric teams organized around business goals
3. Adoption of continuous integration and continuous delivery (CI/CD)
If you’re just getting started with microservices, you can read this Intersect post on how
and why to implement them correctly. For more on building platform-based technology
teams, read this guidance from McKinsey & Company. In this guide, we explore the third
factor: adoption of CI/CD.

The simplest definition of the CI part is that it’s a cycle of regularly checking new code
into a version control system, testing it (via security scans, compliance checks and more)
and integrating it with other code. Companies adopting the CD step also verify that code
works in production and is ready to be deployed at any time. In either case, the steps are
automated to make sure it gets done and to let developers focus their time and energy on
higher-value tasks.

If the goal of agile is to help organizations be highly adaptable to market needs, changing
requirements and new technology, then CI/CD is now a necessity, not a luxury.

1. The Standish Group. “CHAOS Report: Decision Latency Theory.” 2018.

W H I T E PA P E R | 3
The CIO’s Guide to CI/CD

Fast feedback and tight focus

Think about the last time you created something—a presentation, a blog post or even
an internal memo. Perhaps you sought some feedback or a copy edit from a peer or
your boss before you shared it with its intended audience. You probably were most
ready to absorb and incorporate this feedback the moment you hit “send” on the email
asking for review.

Now imagine getting that feedback three months later. First, you have to deal with
switching contexts from whatever else you’ve been working on in the meantime. You’ve
also potentially missed the opportunity for that feedback to have impact: The meeting you
presented in is over, the report has been circulated. Then you realize that it’s not just
about fixing a few typos because the world has changed since then, and you now need
to rework the piece quite a bit to keep it relevant.

It would have been much easier and more efficient to publish it immediately and then
make small changes as necessary—or maybe just write a series of smaller posts to
complement the original. Your frustration probably would be doubled if what you
requested that extra set of eyes on was a small thing to begin with.

And although it seems absurd to imagine dealing with three-month feedback cycles on
your work, it’s unfortunately not uncommon in software development. In fact, some
organizations operate on much slower timeframes:
• At T-Mobile, it took seven months to make a single change on one particular service.
• Research on software procurement in the military found a median of eight years from
initial development to initial operational capability, which the military defines as the state
achieved when a capability is available in its minimum usefully deployable form.
CI/CD can play a big role by helping get those months down to days or even hours.
Change velocity is important precisely because it provides immediate feedback. Teams
are pushing small updates or fixes more frequently, which speeds up feedback loops,
allowing developers to focus on the one thing they are trying to accomplish. Faster
velocity also mitigates risk because you’re not trying to push a stockpile of changes at
once, months after they were coded.

But before we break down the CI and CD parts in more detail, we need to discuss the
importance of testing. “Move fast and break things” might have been a fine motto for
Facebook engineering in 2008, but it’s probably not going to fly for a large enterprise in
a regulated industry. Without an adequate testing protocol—and without understanding
why you’re doing all those tests—moving fast actually will break things.

W H I T E PA P E R | 4
The CIO’s Guide to CI/CD

First: You must write the tests

CI/CD ensures code is ready for deployment at any time. To safely deploy software,
software developers and product managers need to know that it works. Tests are used
to determine if functionality works (unit tests) and that it doesn’t break other parts of the
code base (integration tests). As such, the benefits of CI/CD are, on some level, predicated
on tests.

How many tests do you need? That depends on how quickly you want to iterate. When
code changes infrequently, limited tests and manual testing processes are OK. But as
developers want to release code more quickly, good test coverage is needed to give
rapid feedback on the code’s viability. Teams should produce tests as often as they
produce new code. This is why test-driven development is such a core tenet in extreme
programming practices.

“Tests give you the confidence you need to refactor your code to keep it
clean so that you can go fast forever.”2
MATTHEW PARKER
VMWARE

Without tests, making changes to code is fraught with uncertainty and risk. However,
many companies want to release code faster than they currently do, but don’t have the
test coverage to do so with confidence. Faced with lots of legacy code that is risky to
change, it’s daunting to conceive of writing all those tests.

Writing tests is an investment. It takes time and there are no shortcuts. However, it’s
an investment that, like a sound financial investment, yields increasing benefit over time.
In fact, it is the investment that pays down technical debt. Just as forming a savings
habit takes time and practice to make the behavioral shift begin to feel normal, so does
writing tests.

Continuous integration explained

Once your organization has bought into the value of fast feedback and delivery, as well as
the importance of writing tests, it’s time to step up to the plate of continuous integration.

Continuous integration is a software-development practice where development

teams frequently integrate their code into a shared repository. By integrating code often
(typically, at least daily), developers can ensure their code is not harmful to other parts of
the system.

Each commit of new code is verified by an automated build and a series of test suites. Unit
tests ensure the committed code itself works, while integration tests are invoked to ensure
breaking changes are not introduced into the main code line.

Another benefit of continuous integration is the notion of immediate feedback. By

integrating changes often, developers receive immediate feedback on the quality of the
code they’re delivering. This reduces the cost of fixing issues. Bugs are rapidly found and
fixed within minutes or hours instead of days or weeks.

2. VMware. “Why TDD.” Matthew Parker. March 2, 2018.

W H I T E PA P E R | 5
The CIO’s Guide to CI/CD

Continuous delivery explained

After it has passed all of its unit and integration tests, there are still a number of other
steps required before code is ready for deployment. These include compliance
checkpoints and verification that the code works in a production environment, not just
test and dev. This process is referred to as continuous delivery. The goal of continuous
delivery is ensuring code is ready for deployment into production at any time.

Continuous delivery is not to be confused with continuous deployment, an optional,

subsequent step in which code is automatically deployed to production after going
through the CI/CD process.

The end result of CI/CD is that code is ready for release to production, but when that
release occurs is a business decision. In some cases, continuous deployment is the correct
approach but, in other cases, you may want to release code to production on a different
cadence (say, for example, based on the ability of users to absorb new features without
being overwhelmed).
The important point is that teams that practice CI/CD can release new application code to
production in minutes and when it makes the most business sense to do so, rather than
based on predetermined release windows. This means users get updates and new features
on a more frequent, but also more thoughtful, basis.

CI/CD in action
Often the simplest ways to understand a concept is to examine a real-world example.
To understand CI/CD in a real-world way, we can follow a developer’s routine:

1. Download the latest, integrated software from the central repository.

2. Write the tests for the new feature or capability to ensure the new code works and
doesn’t introduce breaking changes.
3. Code a specific feature or capability.
4. Check in the code and carry out an automated build, which will take the existing code
base and a developer’s changes, and verify they compile and pass tests (including all the
pre-existing tests from other components).
5. Integrate the code into the central code repository once it passes tests and builds on a
developer’s machine.
6. Check in the code, which triggers a CI pipeline to test the new code against other
services and dependencies. As tests fail, development teams are notified and the
pipeline does not complete.
7. If all integration tests pass, a continuous delivery pipeline can pick up the tested code
and run additional checks.
The key takeaway: By integrating regularly, you can detect errors quickly and locate them
more easily.

W H I T E PA P E R | 6
The CIO’s Guide to CI/CD

CI/CD as an organizational change agent

Aside from improving the frequency and purposefulness of code deployment, CI/CD
can also bring benefits at the organizational level. When we talk publicly about workforce
transformation, we often do so in the context of learn by doing. What we really mean by
this is to take something that’s important to the business, drive change around it and, in
the process, generate new value.

Continuous integration with test-driven development is a wonderful place to start the

process of breaking down organizational walls. For starters, it requires developers to
embrace a QA mindset because they must write tests and contribute those tests to
integration suites. However, there can be resistance to this process, particularly if folks
have not embraced agile to the degree of forgoing deadlines.

“What we also see is when deadlines start to loom, the first thing people
stop doing is writing tests, or turn the tests off because they’re slowing
us down. ... However, once you do that, you immediately get into a place
where your tests start to atrophy and then turning them back on is of
course a challenge. ... I think my biggest advice is, if you can get away
with it, don’t ever, ever, ever, ever, ever, ever turn off a test.”3
JONATHAN REGEHR
GARMIN

Integration test suites are then leveraged as part of product builds. This requires
engineers and operations teams to come together to figure out where to host and
execute a product build.

Finally, once all tests have passed and a new build has been created, that artifact can be
deployed to production (either via continuous deployment or a manual step). This requires
the important intersection of security, operations and a host of other teams to coordinate
and deliver that code to production. Ideally, this is another step of the process that should
be automated as much as possible.

“We spent a lot of time on our pipeline and our test automation to make
sure that when we say, ‘Hey, we’re continuously delivering,’ that we have
a record of all that. We know exactly what’s deployed. We know that it’s
ready to go because of all the automated testing that says it’s ready to go.”4
KYLE CAMPOS
CSAA INSURANCE GROUP

As you can see in this simplified description, CI/CD takes what has traditionally been a 12-
to 18-month process involving often hundreds of people and shrinks it to minutes or hours
in an automated, rapid fashion. To achieve this massive feat, organizations must come
together and focus on the primary goal of rapidly delivering value to end users.

3. VMware. “Passing the (Automated) Test at Garmin with Jonathan Regehr.” Jeff Kelly. July 31, 2018.
4. VMware. “Making Transformation Real at CSAA.” Jeff Kelly. March 27, 2018.

W H I T E PA P E R | 7
The CIO’s Guide to CI/CD

How CI/CD affects end users and customers

Beyond introducing new technical efficiencies and organizational improvements, CI/CD
also can have a meaningful impact on the customer experience. This is critical in today’s
software-driven world, where most serious companies focus on the idea of digital
transformation—essentially a fancy way of saying “improving the customer experience
via software.”5

By automating the processes between writing code and releasing it to users, CI/CD lets
you deliver value in small batches. These incremental improvements can be deployed to
production environments, where users begin interacting with the software and provide
valuable feedback. With this newfound velocity, you’re able to embrace the notion of an
MVP (minimum viable product), which leads to the rapid evolution of applications based
on user feedback. We see this play out almost subconsciously in our day-to-day lives with
services such as Netflix regularly tweaking, releasing and/or deleting new features based
on what’s working and what’s not.
Driving adoption is a continuous cycle that doesn’t end after the first release. It’s for
this reason that velocity gains from CI/CD are so critically important and necessary for
enterprises to remain relevant in the face of ceaseless competition from Silicon Valley and
elsewhere. By implementing these work patterns, for example, a major car company is
able to remain innovative, and therefore competitive, despite the fact that it was easier for
Google to learn how to build a car than it was for the already-established car company to
learn to build a self-driving one.

Modern software is all about agility

Although this guide is about CI/CD, it should be noted that almost all new product
development embraces this philosophy. All cloud native products that employ CI/CD
benefit in the form of increased agility and assurance of quality.

We can also infer that companies that embrace CI/CD as a delivery method see software
development as a strategic capability that provides a competitive advantage. Amazon
was a great example when it first publicized the ability to deploy software to production
environments every 11.6 seconds in May 2011. It has since increased its agility, expanded its
system, added more components and so on—and it was all made possible by a relentless
focus on automation, integration and delivery.

But Amazon is not alone: Facebook, Google, Dropbox and all the other tech giants
release software many times a day. Increasingly, they’re being joined by more-traditional
enterprises—such as Walmart, T-Mobile and many, many more—that are committed not
just to remaining relevant in the 21st century, but also to growing. For example:
• Scotiabank deploys more than 3,000 times a month.
• Garmin deploys to production 3,400 times per month.
• Liberty Mutual deploys 1,000 times a day to production on 2,500 daily builds.
You don’t do that by resting on your laurels and expecting customers to adjust their busy
lives around your slow-moving software practices. You do that by getting more agile and
delivering the types of digital experiences your customers and employees demand.

5. VMware. “How CEOs can drive digital innovation without learning to code.” Derrick Harris. March 18, 2019.

W H I T E PA P E R | 8
The CIO’s Guide to CI/CD

Additional reading
Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment
Automation by Jez Humble and David Farley: This is the definitive CI/CD book of its time
and still very relevant today. Learn more about why CI/CD are important foundational
building blocks in any software organization.

2018 Accelerate State of DevOps Report: Dig into the longest-running research on
software delivery performance. It can help you validate your progress and help you
set goals for improvement around the delivery metrics that matter: throughput, stability
and availability.

How to Become a Continuously Delivered Enterprise: Delve into why delivering modern
applications means a new way of thinking about risk, change and roles with Lyle Murphy.

Towards Progressive Delivery: RedMonk’s James Governor explores emerging patterns in

CI/CD, such as feature flagging, blue/green deployments, canary deployments and more.
Speed Thrills: How to Harness the Power of CI/CD for Your Development Team: A
more detailed guide for practitioners covering CI/CD steps, measurement, tooling
and best practices.

A version of this guide also appeared on the Cognizant blog.

About Intersect
Intersect is a publication designed to guide executives and other enterprise decision-
makers through the ever-evolving nexus of business and technology. Through articles,
reports, newsletters and more, Intersect provides the insights and information you need
to keep up with what’s happening in enterprise IT and how it affects your business.

If you’d like to contribute to Intersect, please email [email protected]. You can also
view all of our Intersect posts at tanzu.vmware.com/intersect.

W H I T E PA P E R | 9
VMware, Inc. 3401 Hillview Avenue Palo Alto CA 94304 USA Tel 877-486-9273 Fax 650-427-5001 vmware.com Copyright © 2020 VMware, Inc.
All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents
listed at vmware.com/go/patents. VMware is a registered trademark or trademark of VMware, Inc. and its subsidiaries in the United States and other jurisdictions.
All other marks and names mentioned herein may be trademarks of their respective companies. Item No: The CIO's Guide to CI CD 05/20