W H I T E PA P E R

Microservices Testing
Strategy Guide
 Whitepaper
Microservices Testing Strategy Guide

Microservices have been an industry trend for years now, yet organizations fail to reap the benefits
of the approach and struggle with failed releases. These failures often boil down to the difficulty of
testing the interfaces between services to get the quality, security, and performance that’s expected.

In the end, it’s a failure to test these APIs in a robust enough manner. The silver lining is that the
testing concepts and solutions for legacy SOA testing and microservices testing are the same. Solving
API testing issues means improving microservice releases.

2
 Whitepaper
Microservices Testing Strategy Guide

MICROSERVICES INTRODUCE NEW CHALLENGES

These days, it’s commonplace to see hundreds, if not thousands, of microservices grouped together to
define a modern architecture.

Highly distributed enterprise systems have big and complex deployments. This complexity in the
deployment is often seen as a reason NOT to implement microservices. In fact, the complexity
within the monolith being decomposed is just breaking apart into an even more complex
deployment environment.

To the disappointment of the uninitiated, complexity isn’t really going away. Instead, it’s morphing into
a new kind of complexity.

While microservices promise to increase parallel development efficiency, they introduce their own
new set of challenges. Here are some examples.

» Many more interactions to test at the API layer, whereas before API testing was limited to just
testing externally exposed endpoints.

» Parallel development roadblocks that limit the time-to-market benefits of breaking up the
monolith. Dependencies on other services and complex deployment environments reduce the
parallelism in reality.

» Impacts to the traditional methods of testing that were successful within the monolith, like end-
to-end UI testing, must now shift to the API layer.

» More potential points of failure due to distributed data and computing, which makes
troubleshooting and root cause analysis difficult and complex.

API & MICROSERVICES TESTING

Testing APIs and overcoming the challenges they represent is not that different whether the
operating environment is highly distributed, somewhat distributed, or mostly monolithic. The
greater the number of integrations that exist, the more acute some of the challenges will be, but the
fundamental approaches are the same.

When thinking about an API or microservice as a black box, it can be broken down into the clients of
a service and the dependencies of a service. Testing an API as a black box means the test harness is
acting as a client to the service under test whose job is to verify that it receives the correct responses
back. The dependencies are what the service needs to integrate with to function properly. There are
optimizations to be made on both the client side and among the dependencies.

Before exploring these optimizations, let’s begin at the design phase, where the microservice is
planned and where the testing strategy should begin.

3
 Whitepaper
Microservices Testing Strategy Guide

THE MICROSERVICE LIFE CYCLE

The life cycle for microservices software development is like that of other software. Assuming a
typical CI/CD pipeline is in place, development transitions from planning to design to implementation,
and then to testing. During these phases of development, the use of best practices helps improve
microservices quality and security.

Plan

Deploy Delivery Design

Figure 1:
Microservices software
development life cycle
Testing Implementation

DESIGN PHASE: DEFINE REQUIREMENTS FOR CLARITY

A well-accepted API development best practice is to implement a service definition during the design
phase. The OpenAPI specification is commonly used for RESTful services.

Clients use these service definitions to understand what resources and operations a service
supports and how this service expects to receive and send data. Within the service definitions for
REST services will be JSON schemas that describe the structure of the message bodies, so client
applications will know how to work with an API.

But service definitions aren’t just important for helping other teams understand how an API works.
They also have positive impacts on a testing strategy.

Contracts

A good way to think of a service definition is like a contract. That’s the basis to start building a
strategy for API governance. One of the biggest challenges in API testing, just like with any software
testing, is dealing with change.

4
 Whitepaper
Microservices Testing Strategy Guide

Change has never been more rapid than with the popularity of Agile practices. Enforcing these
contracts is the first step in bringing order to chaos when trying to wrangle many teams that are
building APIs and then expect all these APIs to somehow be compatible with each other.

Validating & Enforcing Contracts

The challenge is how to enforce a contract. As part of any automated regression suite, there should
be checks to see whether the service definition as written has any mistakes in it. Swagger and
OpenAPI have schemas of their own that are useful to define how the service definition must be
written. These can be used to automate these checks early in the API development life cycle.

Then, besides verifying the contract itself, the next step is to check that the responses from the
service under test conform to the contract. A suitable API testing framework should have support
built in to catch instances where the API is returning a response that deviates from its service
definition’s schema.

Think of it like this: A car is made up of thousands of individual parts that all need to fit perfectly.
If, during car assembly, the team responsible for the power unit delivers an engine that deviates
from the design spec, it will likely cause big problems when hooking it up to the transmission that a
different team built because they were referencing the engine’s design spec to know where the bolts
need to line up.

That’s what is being tested for here. These are the kinds of integration problems that good API
governance can help avoid. Designing by and conforming to contracts should be one of the first
concerns of any API testing practice.

Service definition contracts can also help the testing process be more resilient to change. The time it
takes to refactor test cases for the changes in an API could have a huge impact on testing such that
it gets pushed outside of the sprint.

When using an API testing framework, it needs to help teams bulk-refactor existing test cases as an
API’s design changes so they can keep up with the fast pace of Agile and in-sprint testing. Service
contracts and the right tooling can make this much less painful.

IMPLEMENTATION PHASE: APPLY BEST PRACTICES

Microservices development doesn’t imply a free pass to skip unit testing. Code is code and unit
testing is a fundamental quality practice. It helps teams catch regressions quickly, early, and in a
manner that’s easy for developers to remediate—no matter the kind of software.

The ugly truth is that software teams with nonexistent or reactive unit testing practices tend to
have poor quality outcomes. This is the result of the high overhead of a unit testing practice, and
therefore many managers and leaders don’t prioritize it.

This is unfortunate because the market for tools has matured to the point where a developer’s life
is much easier and more productive while unit testing, and the trade-off between cost and quality
is much less than it used to be. Unit testing forms the foundation of a solid testing practice and
shouldn’t be shortchanged.

5
 Whitepaper
Microservices Testing Strategy Guide

In addition, software quality practices for developers have matured with dedicated coding standards
for API development. In 2019, the international nonprofit, OWASP, released the OWASP Top 10 API
Security standard.

Coding standards like this help microservices teams avoid common security and reliability anti-
patterns that could introduce business risk to their projects. Trying to adopt a coding standard
without tools is nearly impossible.

Luckily, modern static analysis tools, also known as static application security testing (SAST) tools,
are staying current with industry standards and will have support for this standard. Developers can
scan their code as they’re writing it as part of their continuous integration process to make sure
nothing gets missed.

COMPONENT TESTING PHASE: USE PROXIES FOR API DEPENDENCIES

Component testing implies testing a microservice in isolation. Achieving true isolation is challenging
because of the numerous dependencies most services have. Furthermore, one of the most difficult
things to anticipate as a microservice developer is understanding exactly how an API is going to be
used by other systems.

Client applications of an API are likely to find creative uses of the microservices that were never
considered. This is both a business blessing and an engineering curse, and exactly why it’s important
to expend energy to understand the use cases for an API.

API governance during the design phase is an important first step. But even with well-defined
contracts and automated schema validation of a microservice’s responses, it’s not possible to fully
predict how the needs of the end-to-end product are going to evolve, and how that will impact the
microservices within an organization’s domain.

Following are the three key basic steps to microservices testing.

1. Record

2. Monitor

3. Control

RECORD MONITOR CONTROL

Figure 2:
Three key basic steps
to microservices testing

6
 Whitepaper
Microservices Testing Strategy Guide

Recording, monitoring, and controlling help implement testing approaches that efficiently discover
what tests to create while simultaneously helping to automate component tests when there are
downstream APIs that need to be isolated. The enabling technology to do this is service virtualization.

Service Virtualization

Service virtualization is a function of software testing that can emulate an application’s

dependencies. It allows DevOps teams to use computer-generated virtual services in place of real
ones so they can test often, test early, and lower costs in various situations. These include key parts
of the distributed system that are missing. An example of this is when dependent components are
inaccessible or lack the necessary data or capacity.

Service virtualization has also been proven to cost effectively scale training/sales environments and
partner onboarding scenarios. Using virtual services for shift-left or in-sprint testing is more cost-
effective than trying to scale fully integrated “real” environments to meet the increased demand.

Consider how expensive your performance environments are to provision and maintain for testing.
Automated testing with endpoint simulation provides the following benefits.

» Eliminates the chances of losing data.

» Reduces the number of expensive servers needed for the end-to-end system to operate.
» Allows the company to forgo excess license fees.
The result? Fast and accurate testing procedures that are primed for Agile and DevOps workflows.

Message Proxy

Message proxies, made possible with service virtualization technology, monitor and record the
message flows between APIs, as well as control the destinations where messages are sent.

A message proxy is designed to listen to a given endpoint or queue, and then forward messages it
receives to a destination endpoint or queue. Basically, it’s a man-in-the-middle for APIs. Proxies can
be injected as needed between services in already running systems. Once it’s in place, it’s ready to
start working.

7
 Whitepaper
Microservices Testing Strategy Guide

Authentication Monitoring
& Security Command-line

CI Message Proxy Execution

Message
Integration Routing
Data
Reuse
Clustering/ Scriptless
Scaling SOAP
API
Key Features
of Service AI Powered
Data-driven
Asset
Responses Virtualization Creation
EC2
Azure REST Record &
API Playback

Governance
Service
Templates Cloud
Support
Stateful Test Data
Behavior Re-use Management/
Emulation Services Generation

Figure 3:
Key Features of
Service Virtualization
Message proxies make use of the monitoring, routing and record, and playback features of service
virtualization to perform this go-between capability.

Message
API Client API Endpoint
Proxy
Orchestration Choreography

HTTPS

Event Broker
MQ
VS

Figure 4:
A message proxy sits between clients and endpoints
for recording, monitoring, and controlling API traffic.

8
 Whitepaper
Microservices Testing Strategy Guide

Orchestrated vs Choreographed Services

Orchestrated and choreographed (or reactive) services are used to describe synchronous or
asynchronous messaging patterns. If a service is communicating using a message broker with
protocols like AMQP, Kafka, or JMS, then it’s a choreographed or reactive service. If testing includes
REST or a GraphQL interface, then it’s an orchestrated, or synchronous, service.

The techniques outlined in this paper apply regardless of the protocol and message exchange pattern
used. However, it’s more difficult to apply these principles to asynchronous messaging if the chosen
API testing solution doesn’t have support for the protocols used. In other words, support for API
protocols is an important factor in selecting API testing solutions.

Capture Client Usage Scenarios

It’s difficult for a microservice team to predict how other teams will use their API. End-to-end, fully
integrated testing is expensive and slow. Using the message proxy capabilities found in service
virtualization tools to record traffic coming from upstream client applications captures realistic usage
scenarios. These scenarios help improve knowledge of what tests should be run within the CI/CD
pipeline without requiring those client applications to always be present to trigger that traffic.

In other words, the recording enables the replay of these integration scenarios for automated
regression testing. That’s a lot simpler and easier to manage than asking clients to run their tests to
transitively test your API. This is why tools that combine service virtualization with API testing are so
popular. They make it easy to record this API traffic and then leverage it for API scenario testing that’s
brought under control.

API Test
Figure 5: Scenario
Clients generate API
traffic that is recorded
by message proxies. This
traffic is organized into
API test scenarios.
Make Dependencies Manageable

Testing services quickly becomes difficult because of their reliance on other APIs in a test
environment, which brings issues with availability, realistic test data, and capacity.

This can push the testing effort out of the sprint and make it difficult for teams to detect integration
issues early enough that they have time to deal with them. This is the traditional use case for service
virtualization, where the technology allows for simulating or mocking the responses of downstream
APIs (in so doing, creating a virtual version of the service), providing isolation so that it’s possible to
fully test an API earlier and easily within a CI/CD pipeline.

9
 Whitepaper
Microservices Testing Strategy Guide

!
Application
Under Test

QA and Test
Team A’s API

Team B’s API

Development
Team C’s API

Performance
Test Engineer Costly or parallel deployments block test automation

Test data challenges impact testability

Capacity limitation hinders performance testing

Figure 6:
Trying to test a
microservice in its real When message proxies are deployed in an environment, teams can record API traffic and then
environment is difficult build virtual services that faithfully and realistically respond to the microservice—including stateful
due to the lack of parallel
transactions where the simulated dependency must properly handle PUT, POST, and DELETE
deployment, test data
limitations, and lack of operations—without having the real dependency available.

!
capacity.
Stateful service virtualization is an important feature to create the most realistic virtual service that
covers more use cases.

Application
Under Test

QA and Test
Virtual ‘A’ Service

Virtual ‘B’ Service

Development
Virtual ‘C’ Service

Performance
Test Engineer

Figure 7:
Service virtualization
decouples the service
under test from the
operational environment
while still providing a
realistic test environment.

10
 Whitepaper
Microservices Testing Strategy Guide

INTEGRATED TESTING PHASE: CONTROL THE TEST ENVIRONMENT

Let’s zoom out the testing scope a bit further to integration testing. At this stage, sometimes
referred to as the system-integration-test or SIT stage, the testing environment is a production-like
environment that ensures there aren’t any missed defects.

The expectation is that message proxies provide control over the choice of isolated or real-world
connection to dependencies. Another aspect of control is making sure the message proxies can
easily be managed via API. Organizations with high maturity in their CI/CD processes are automating,
deploying, and destroying workflows where the programmatic control of the message proxy is a
must-have requirement.

The monitoring capabilities are essential for a service virtualization solution that supports automated
testing. The monitoring from message proxies will expose the inner workings of these complex
workflows to support the implementation of better tests that pinpoint where problems lie, not just
that a problem exists.

Consider, for example, an order processing system that needs to check multiple downstream services
like inventory, billing, and shipping to fulfill an order. For a given test input, an assertion against
certain behind-the-scenes behavior helps developers pinpoint why a test is failing. When a team
spends less time figuring out why a problem occurred, they have more time to actually fix it.

!
EDI Products / Billing /
Partner Purchase Order
Check inventory for Shipping
Integration Workflow
each product in PO Microservices
HTTPS

In Stock

In Stock

In Stock

Out of Stock

Figure 8: In Stock
Service virtualization
allows for the assertion
of behavior to help test
or debug scenarios that
would be difficult to
replicate in an operational
environment .

11
 Whitepaper
Microservices Testing Strategy Guide

FURTHER APPLICATIONS
Modern mobile and web UIs are built on top of APIs. Herein lies another opportunity to capture
API traffic from UI tests to help drive API scenario testing. This frontend API may not be well
documented or well tested compared to the core services in the architecture, but that’s even more
reason to test them!

Translating an end-to-end UI test into its corresponding API tests can be a daunting task to perform
manually. When using the right tool, AI can significantly reduce the time investment on this task.

APIs have become the bedrock of how organizations deliver functionality to their users quickly. But
functionality isn’t the only concern that keeps company management up at night.

Resilience, performance, and security are all nonfunctional requirements that continue to capture
the public’s attention with scary headlines. Many organizations have shift-left initiatives to try
and reorganize their software delivery processes to account for these high priorities. Using a
framework that’s well suited to cover both functional and nonfunctional testing without the need to
conglomerate half a dozen tools together will cut down on testing overhead.

SUMMARY
Microservices are here to stay. Unfortunately, organizations are failing to reap the benefits of the
approach due to the difficulty in testing the interfaces between distributed systems—the APIs—to get
the quality, security, and performance that’s expected.

The solution is an approach to microservices testing that enables the discovery, creation, and
automation of component tests. The enabling technologies are message proxies and service
virtualization, which are tightly integrated with a feature-rich API testing framework.

Message proxies enable recording of API traffic, monitoring to discover scenarios and use cases, and
control to manage and automate suites of API tests. Coupled with service virtualization, automated
microservice testing has become an achievable reality.

12
 Whitepaper
Microservices Testing Strategy Guide

TAKE THE NEXT STEP

Request a demo to see how your development team can seamlessly validate microservices with an
automated, tightly integrated testing approach.

ABOUT PARASOFT
Parasoft helps organizations continuously deliver quality software with its market-proven, integrated
suite of automated software testing tools. Supporting the embedded, enterprise, and IoT markets,
Parasoft’s technologies reduce the time, effort, and cost of delivering secure, reliable, and compliant
software by integrating everything from deep code analysis and unit testing to web UI and API
testing, plus service virtualization and complete code coverage, into the delivery pipeline. Bringing
all this together, Parasoft’s award-winning reporting and analytics dashboard delivers a centralized
view of quality enabling organizations to deliver with confidence and succeed in today’s most
strategic ecosystems and development initiatives—security, safety-critical, Agile, DevOps, and
continuous testing.

13