BROUGHT TO YOU IN PARTNERSHIP WITH

CONTENTS

öö Introduction

Design Patterns for

ö ö What Are All These Continuous
Practices?

ö ö What’s Wrong With Continuous

Continuous Delivery:
Delivery?

ö ö DPAT: The Continuous Release

Cycle

ö ö Benefits of Continuous Releasing

for Software and Business

Restructuring DTAP for DevOps ö ö Conclusion

ö ö Summary

ö ö Key Takeaways for Practicioners

WRITTEN BY OLAF MOLENVELD CTO & CO-FOUNDER OF VAMP

AND JOEP PISCAER ARCHITECT AND PATHFINDER

You will learn:

Introduction
Organizations are adopting microservices and DevOps to stay com- 1. How testing is handled in continuous integration, delivery, and
petitive and increase the speed of releasing new functionalities with deployment
improved scalability, quality, and (cost) efficiency. 2. The promise of these three “continuous practices”
3. How continuous delivery falls short on these promises in practice
Current processes, tooling (continuous integration/testing/packaging/de-
4. How DTAP stands in the way of continuous delivery fulfilling its
ployment), and Development/Test/Acceptance/Production environments
promise
(DTAP) are typically designed for traditional codebases and applications,
5. How to restructure DTAP for continuous releasing to gain the advan-
manual and time-consuming testing, separate organizational silos and
tages of continuous delivery for software and business
handovers, and “big bang” deployments to production.

With the move to DevOps, these traditional environments, processes, and What Are All These Continuous Practices?
Various practices have been developed to optimize parts of the software
tools no longer serve in reducing risk in the software delivery lifecycle
development and delivery process. In this Refcard, we’ll consider each
(SDLC.) Instead, they act as a block to the full potential of increased effi-
specifically as it relates to testing.
ciency in software delivery. 

Because of the DevOps mantra of “you build it, you run it,” development
teams are allowed a lot of autonomy and control, and are expected to also
handle quality assurance (QA) and releasing tasks. The reality is that these
are specialized and complex tasks that need expertise and dedicated tool-
ing provided to software delivery teams. In addition, business outcomes
are ignored as they are hard to validate for development teams, and QA
becomes mainly focused on technical aspects.

In this Refcard, we’re going to give you a guide on how you can rethink
your software delivery methodologies for modern software development
— one that reinjects the business function of QA and takes advantage of
continuous releasing in order to lower the risk of releases, shorten devel-
opment cycles, and help software support business.

1
 DESIGN PATTERNS FOR CONTINUOUS DELIVERY

CONTINUOUS DEPLOYMENT
A Note on Terminology:
Continuous delivery and continuous deployment are very similar con-
In this Refcard, we’ll be using terminology with some overlap. To prevent cepts. Where continuous delivery promises deployable code, potentially
confusion, let’s define it here. put into Production, continuous deployment always automatically puts
the code into Production.
The acronym DTAP is short for Development, Testing, Acceptance, and
Production. These four separate environments are used in a phased The technical act of deploying involves copying deployable code to the

approach of software development. Production environment so that it is ready to receive user traffic.

The technical act of releasing means making the features that code
In this guide, we refer to each of these environments using capitalization:
implements available to users.
Development, Testing, Acceptance, and Production.

However, whereas deployment remains only a technical act, releasing

• A Development environment is often a developer’s laptop.
has both a technical and business — decision-making — function in the
• A Testing environment is where completed and built code is tested
process of software delivery.
to verify it works as expected. Testing should closely resemble
Production. THE CONTINUOUS PROMISE
• Acceptance is where successfully tested code is deployed for final Each of these continuous practices has evolved to optimize parts of the
verification, sometimes involving the end user in a pilot group. software delivery process. Together, they promise:
• Production is the environment where code is made available to
Quality of code. The primary goal of continuous integration is to make
all users.
sure that code works and doesn’t regress between releases.
In addition to the environments, we refer in lower case to development and
Low risk releases. The primary goal of continuous delivery is to make the
testing as the processes where engineers go through different phases of work.
deployment of new software painless.

CONTINUOUS INTEGRATION Faster time to market. The overall goal of the continuous practices is put-
Continuous integration, or CI, is a set of practices during development ting software to work as quickly as possible to achieve business goals.
of code.
What’s Wrong With Continuous Delivery?
CI is essentially the automation of a set of housekeeping rules for develop- In the ideal world, CI/CD promises to shorten the feedback loop between
ers that makes sure that the Development environment and code repos- Production and the development process and to allow developers to
itory remain in shape. The basic practices are to use version control (like optimize performance without long wait times or switching context. This
Git) and to regularly check in all code. Developers make sure that the code is not surprising, since 70% of digital transformation projects focus on
they check in passes quality tests and builds into an artifact successfully. optimizing CI/CD tooling.

The promise of CI is that work from individual developers moves into In practice, the DevOps approach still has shortcomings. Testing is
Testing more quickly and without major coding errors, freeing up devel- focused on automated testing — on technical checks of developers’
opers and testers to pick up new work. output (the code) — rather than checking for improvements in business
outcomes (conversion, revenue) or impact (market share, profitability,
The focus of the testing done during the integration phase is on high customer satisfaction).
code quality.
Relying on automated testing in CI/CD leaves behind exploratory testing. As
CONTINUOUS DELIVERY a result, the net effect of many CI/CD efforts is that business-oriented QA
Where CI makes sure that the code in the repository is always ready to move engineering is cut out by automating just the low-hanging fruit in CI/CD;
to the next environment, continuous delivery, or CD, is a set of practices that often skipping out on the more valuable, but more complex QA testing.
builds on CI to make sure code is always deployable to Production.
Or even worse, Testing and Acceptance are skipped entirely by letting
CD implies the automation of all steps needed to take a built artifact and developers write code and put it into Production based on developer-led
deploy it into Testing, Acceptance and Production environments. testing, cutting out the checks and balances QA would provide. (This is
inefficient, as it results in 20-50% of developer time wasted on non-func-
The promise of CD is to have a pipeline that completely automates the
tional tasks.)
deployment (and infrastructure provisioning and configuration).
This happens because there are systemic issues with Test and Accep-
The tests done in the CD pipeline are focused on catching regressions tance environments of a typical Development-Testing-Acceptance-Pro-
and checking performance before moving to Production. duction cycle.

3 BROUGHT TO YOU IN PARTNERSHIP WITH

 DESIGN PATTERNS FOR CONTINUOUS DELIVERY

While Testing and Acceptance are great for doing many technical tests, This type of testing is called canary testing, and is not unlike the small-scale
some testing simply requires real users. No amount of synthetic testing trials of testing new versions of software with a group of pilot users.
in pre-Production, automated or otherwise, can identify all the potential
The major difference, however, is that scaling up a canary release is
causes of user experience degradation.
much more granular and controlled, as QA engineers can tap into a

TRADITIONAL DTAP DOESN’T WORK FOR DEVOPS larger array of conditions and users to segment traffic and immediately
Since we’re no longer building traditional monolithic applications using see results in business outcomes.
traditional methods and silos, we cannot use static pre-production Test
SETTING CONDITIONS TO SEGMENT USER TRAFFIC: AUTOMAT-
and Acceptance environments to ensure our application will work as
ING CONTINUOUS RELEASING
intended. This is because of the independent nature of microservices This is what we call continuous releasing: the validation of new versions
and their deployments. of software early in the software delivery lifecycle — in Production — by
releasing them gradually to a small segment of user traffic. By testing
new versions of a service on real user traffic, QA engineers can validate
both the technical validity of a release and improve business outcomes,
such as conversion, revenue, or basket size.

Continuous releasing allows teams to validate software and business out-

comes by giving them control over what is released to users. That control

Representative QA engineering that traditionally takes place in the comes through setting release policies. These policies govern the condi-

Testing and Acceptance environments should be done on real user traffic tions under which a new version is released to users, and in what steps.

in Production.
Conditions can be as simple as a percentage of traffic, or as complex as a

The solution is rearranging the release train so that the business-oriented selection of users based on location, device type, or login status, as well

QA engineering “acceptance and testing” steps occur in Production. as business conditions such as “has ordered Product X before.”

This gives back control to the QA engineer, and once again opens up Policies allow QA engineers to automatically test an ever-increasing set
the opportunity of testing in a meaningful way: in Production, with of technical and business criteria to optimize new software for business
segments of real user traffic, to validate new features and optimize the outcomes such as conversion, basket size, or revenue in Production.
business value of software.
Benefits of Continuous Releasing for Software and
Business
DPAT: The Continuous Release Cycle
Continuous releasing extends the promise of continuous delivery to
DPAT stands for Development, Production, Acceptance, and Testing. It is
improvements in business outcomes by re-inserting quality assurance at
a reordering of the traditional DTAP. The rationale behind DPAT is simple:
the end of the release pipeline, where it can now be carried out safely in
QA testing yields the best results when done in Production. In fact, it is
Production.
only by moving quality assurance to Production that QA can validate
new features and optimize the business value of software. Continuous delivery and continuous releasing are not mutually exclu-
sive. In fact, they complement each other.

CI/CD works well to support development work, allowing developers

to create better quality code by giving them the tools to automate an
ever-growing battery of technical tests to improve software quality, secu-
rity, and performance issues.

Continuous releasing, on the other hand, supports the work of quality as-
surance, giving QA engineers the tools they need to execute business-ori-
The right architecture allows control over releases ented tests to improve both technical and business outcomes for software.

Unlike past architectures, microservice-style architectures now allow

Conclusion
this type of business-focused representative QA testing to be done in
In conclusion, the continuous releasing practice treats quality assurance
Production. Having loosely coupled, independent services allows teams
as a business function, responsible and accountable for releasing new
to release multiple versions of the same service and granularly control
software. This delegates decision-making about feature development
the flow of traffic to each version. This allows teams to validate the qual-
back into the hands of the business, giving product owners — the
ity of each version independently in a gradual and controlled way, under
stakeholders most qualified to make business decisions — control over
real production conditions.
releases, closing the software delivery lifecycle.

4 BROUGHT TO YOU IN PARTNERSHIP WITH

 DESIGN PATTERNS FOR CONTINUOUS DELIVERY

2. Distribute separate roles and responsibilities for both the deploy-

Summary
In this Refcard, we’ve examined a number of fundamental changes you ment stage and the releasing stage of your SDLC

can make to your software delivery lifecycle to gain the competitive advan- • Assign DevOps engineers control over the technical de-
tage of microservices and DevOps and lower the risk of releases, shorten ployment stage and the first step of the releasing stage that
development cycles, and help software support business. involves technical validation.
• Assign control over business validation to quality assurance
We’ve introduced the concept of restructuring DTAP as DPAT in order to
and product owner roles.
create a continuous release cycle. By moving quality assurance testing
to the right — into Production — teams can gain the advantage of low 3. Move business-focused QA testing to the right, into Production

risk releases for software and organizations can gain the advantage of Rearrange your release train so that the business-oriented QA

increased time-to-value for business. engineering steps occur in Production, after the technical act of
deploying services.
Key Takeaways for Practitioners 4. Develop automated release policies
According to the DevOps Research and Assessment (DORA) organiza- In conjunction with stakeholders, define straightforward policies
tion’s 2018 State of DevOps Report, successfully evolved DevOps compa- for releasing new software based on stakeholder objectives. Codi-
nies are 44 times more likely to have repeatable and automated software fy these objectives as Service Level Objectives or SLOs.
configuration and release automation. After speaking to more than 100
5. Automate Service Level Objectives and Indicators
enterprises over the years about their DevOps journey, we’ve distilled
Define the health and success factors for releases to enable early
our best practices into these steps for increasing DevOps ROI:
detection and real-time mitigation of technical issues, such as
bugs and/or degraded customer experience.
1. Add a continuous releasing stage after continuous delivery
That will allow development teams to independently validate Typically, introducing a Continuous Releasing stage will double key KPIs,
their code under real world production conditions and observe such as time-to-market, in a few months after implementation. If you are
that other services are not impacted by a new release. interested in learning more, we offer an example business case builder.

Written by Olaf Molenveld, CTO & co-founder of Vamp

Olaf has over 20 years of experience in IT architecture and management roles. While working as an enterprise architect, he
helped teams design, build and release innovative online and e-commerce focused platforms for digital enterprise. In his role
as CTO of Vamp.io, he channels that experience into his vision for a DevOps, microservices and container space where testing
in production and continous release allow for flawless software delivery.

Written by Joep Piscaer, Architect and Pathfinder

Joep is a seasoned IT professional, with 10+ years experience as a CTO, head of IaaS and infrastructure, enterprise architect.
In his most recent role, he worked as a technical pathfinder, straddling the roles of CTO and coach for scrum teams. For his
continued efforts to share, contribute and evangelize in the DevOps space, he received the Veeam Vanguard ‘15 - ‘17, Nutanix
Tech Champion ‘14 - ‘18, the EMC Elect ‘15, Atlantis Community Expert ‘16–’17 and the VMware vExpert ‘09, ‘11 - ‘17 (and
vExpert NSX in ‘16 - ‘17) awards.

Devada, Inc.
600 Park Offices Drive
Suite 150
Research Triangle Park, NC

888.678.0399 919.678.0300
DZone communities deliver over 6 million pages each month to
more than 3.3 million software developers, architects, and decision Copyright © 2019 Devada, Inc. All rights reserved. No part of this publi-
makers. DZone offers something for everyone, including news, tu- cation may be reproduced, stored in a retrieval system, or transmitted,
torials, cheat sheets, research guides, feature articles, source code, in any form or by means electronic, mechanical, photocopying, or
and more. "DZone is a developer’s dream," says PC Magazine. otherwise, without prior written permission of the publisher.

5 BROUGHT TO YOU IN PARTNERSHIP WITH