0% found this document useful (0 votes)
22 views21 pages

The Role of Testing in A DevOps Transformation

The 2019 Tricentis Survey Report explores the evolving role of testing within DevOps transformations, highlighting that testing is critical for success and correlates positively with DevOps maturity. The survey reveals that while organizations are increasing their test automation efforts, many still struggle to scale effectively, with significant differences in how testers allocate their time based on the maturity of their DevOps practices. Key findings indicate that as DevOps maturity increases, testers focus more on strategic tasks, but face ongoing challenges such as test maintenance and orchestration.

Uploaded by

hcsaini.kkr
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
22 views21 pages

The Role of Testing in A DevOps Transformation

The 2019 Tricentis Survey Report explores the evolving role of testing within DevOps transformations, highlighting that testing is critical for success and correlates positively with DevOps maturity. The survey reveals that while organizations are increasing their test automation efforts, many still struggle to scale effectively, with significant differences in how testers allocate their time based on the maturity of their DevOps practices. Key findings indicate that as DevOps maturity increases, testers focus more on strategic tasks, but face ongoing challenges such as test maintenance and orchestration.

Uploaded by

hcsaini.kkr
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 21

E X P E C TAT I O N S V S .

R E A L I T Y :

The Role of Testing in a DevOps


Transformation

2019 SURVEY REPORT


2019 Tricentis Survey Report

Table of Contents

E X ECU TIV E SU M MA RY
Purpose and Methods. . . . . . . . . . . . . . . . . . . . . . . . 3
Demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Summary of Key Findings. . . . . . . . . . . . . . . . . . . . . 3

BREAK D O WN O F KEY FINDINGS


The Relationship between DevOps
Maturity and Test Automation. . . . . . . . . . . . . . . . . 4
Where Testers Are Spending Their Time . . . . . . . . 8
Lifecycle Models and Testing Practices. . . . . . . . . 12
How Agile Adoption Has Affected
Software Testing. . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
The Changing Face of Testing. . . . . . . . . . . . . . . . . 13
Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Appendix 1: Regional Comparison
of Key Survey Results . . . . . . . . . . . . . . . . . . . . . . . 19

2
2019 Tricentis Survey Report

Expectations vs. Reality: The Role of


Testing in a DevOps Transformation
As organizations shift toward agile and DevOps,
testers’ tasks, priorities, expectations, and ability
to meet business needs must evolve. While the full
extent of this evolution remains to be seen, one
thing is clear: the role of testing is changing.

EXECUTIVE SUMMARY

Purpose and Methods Demographics


Tricentis and TechWell partnered on a survey to We surveyed testing and testing leadership roles
understand how the role of testing is evolving at a wide range of enterprises across industries
against the backdrop of modernizing development and regions. We’ve called out notable regional
and delivery methods. The survey also examined differences in Appendix 1.
how these changes affect where testers spend
their time and the scope of their work, as well as
their status, job satisfaction, and ability to meet
expectations across the organization.

Summary of Key Findings


• Testing is a critical element of successful DevOps transformation
• More advanced test automation correlates with greater DevOps maturity
• Test automation levels are expected to increase in the next 12 months
• How testers spend their time evolves significantly as DevOps maturity increases
• Manager satisfaction improves with increased levels of test automation and DevOps maturity

3
2019 Tricentis Survey Report

BREAK DOWN OF KEY FINDINGS

The Relationship between DevOps


Maturity and Test Automation

Testing is a critical element of successful DevOps transformation


In the early days of DevOps, testers often expressed apprehension and uncertainty with regard to how
their jobs would be affected by continuous development and delivery. As DevOps transformation initiatives
become common across organizations, it has become apparent that even with increased reliance on
automation, testers play a critical role in everything from setting testing strategy and assessing release
readiness to managing the evolution of how quality is defined. While that definition continues to evolve,
one thing is clear: testers are critical to the success of DevOps initiatives. In fact, the survey shows that
organizations that have successfully completed DevOps transformation are more likely to prioritize testing.

We asked respondents how far along they were with DevOps transformation and found that progress
towards transformation was positively correlated with the level of importance placed on testing. In
organizations where DevOps transformation is complete, 87% of respondents said testing is critical to
successful DevOps transformation, compared to 49% of testers in organizations that haven’t undergone
DevOps transformation and it isn’t a priority.

How perceived importance of testers changes with degree of DevOps maturity

Extremely Moderately Slightly Not


Important to Important to Important to Important to
Transformation Transformation Transformation Transformation

DevOps transformation is complete 87% 13% 0% 0%

DevOps transformation is in progress 78% 20% 1% 1%

DevOps is a priority, but we have


not started
75% 22% 3% 0%

DevOps is not a priority 49% 23% 6% 6%

4
2019 Tricentis Survey Report

We also noted that testers were significantly more likely to feel that their businesses prioritized testing
in organizations that have completed DevOps transformation. Out of these respondents, 80% said the
business prioritizes testing, compared to 26% in organizations where DevOps isn’t a priority.

The finding that organizations who have successfully completed DevOps transformation are 4x more likely
to prioritize testing indicates that testers are a crucial contributor to the DevOps transformation process.

Does the business prioritize testing as part of the DevOps effort?

Yes No
80
80%
70

60
57% 56%
50
Percentage of 46%
Respondents 40 42% 41%
30
26%
20

10 13%

0
DevOps transformation DevOps transformation DevOps is a priority, DevOps is not a priority
is complete is in progress but we have not started

More advanced test automation correlates with greater DevOps maturity


As expected, the survey results show a positive correlation between DevOps maturity and level of test
automation. A majority of organizations (58%) practicing DevOps at any maturity level are automating at
least 20% of their test suite, and more than one fourth are automating at least 50%.

In striking contrast, of those organizations that haven’t prioritized DevOps, a majority (63%) have automated
less than 10% of testing effort and a mere 3% have reached 50% automation or more.

Still, not all DevOps organizations are successfully scaling test automation. In organizations where
DevOps transformation is complete, a significant number (27%) report automating just 10% or less of
their testing efforts—the same average rate reported by organizations where DevOps transformation is
in progress.

This finding is corroborated by the findings of the 2019 Gitlab Global Developer Survey [1], which found that
despite the increased delivery speed that DevOps delivers, most DevOps teams encounter the majority of
delays during the testing stage. Half of those surveyed called out testing as the biggest source of delay in the
development process, suggesting that scaling test automation is a challenge even for mature DevOps teams.

5
2019 Tricentis Survey Report

The relationship between DevOps maturity and level of test automation

DevOps DevOps DevOps is a priority,


Level of test DevOps is not a
transformation is transformation is in but we have not
automation priority
complete progress started
0% 7% 4% 6% 20%
1% to 10% 20% 23% 28% 43%
11% to 20% 13% 17% 28% 6%
21% to 30% 13% 18% 25% 14%
31% to 40% 7% 10% 6% 6%
41% to 50% 7% 8% 3% 6%
More than 50% 33% 20% 3% 3%

Test automation levels are expected to increase in the next 12 months


The survey shows that a majority of organizations are doing at least some test automation; however,
average levels of test automation within organizations remains low. More than half of respondents (52%)
have automated less than 20% of their total testing effort.

This is similar to the finding in the 2017-2018 World Quality report [2] that indicates average test automation
levels are around 20%, as well as the current year’s report [3] that found rates are between 14% and 18% for
specific test automation activities like functional, API, and performance testing.

While organizations are not significantly increasing their rates of test automation, most are at least testing
it out, so to speak. A 2018 TechWell and Tricentis survey report, The Evolution of Test Automation [4], found
that 72% of organizations in the Americas were doing at least some level of automated testing. A little over a
year later, our survey showed that 80% of respondents are automating some portion of their testing effort,
suggesting a moderate year-over-year increase.

Comparison of testing practices 2018–2019

100 2018 2019

80 85%
80%
76%
72%
60
Percentage of

4
55% 54% 56% 56%
Respondents
40

34%
20
21%

0
Automated software Agile DevOps Continuous Behavior-driven
testing methodologies integration development

6
2019 Tricentis Survey Report

The 2018 survey also found that 24% of respondents were automating more than 50% of testing effort while
44% said they expected to reach that level of automation in the next 12 months. Our 2019 survey shows
only 14% of respondents in the Americas (and 19% on average across all regions) actually automate at least
50% of testing effort. This indicates that the rate of test automation within organizations isn’t growing as
rapidly as many testers anticipated, suggesting that expectations for scaling test automation do not always
match the (often difficult) reality.

Percentage testing effort that is automated

0% 8%
1% to 10% 28%
11% to 20% 16%

5
21% to 30% 18%
31% to 40% 9%
41% to 50% 6%
More than 50% 14%

0 5 10 15 20 25 30
Percentage of Respondents

In this year’s survey, 30% of respondents in the Americas (and 35% on average across all regions) expect
that their organizations will automate more than 50% of testing in the next 12 months — a hopeful, but
slightly more modest, expectation.

Percent of automated testing effort expected in the next 12 months

0% 3%
1% to 10% 13%
11% to 20% 11%
21% to 30% 16%
31% to 40% 15%
41% to 50% 12%
More than 50% 30%

0 5 10 15 20 25 30
Percentage of Respondents

Together, these findings suggest that testers and testing leaders understand that an advanced level of test
automation is critical for transformation, but that achieving it is more difficult than anticipated.

7
2019 Tricentis Survey Report

Where Testers Are Spending Their Time

To get a better idea of why organizations struggle to scale test automation to the desired level, we asked
testers where they’re spending the most time today, what their biggest pain points are, and where they have
to make tradeoffs. We also looked at how these responses differ according to whether their organization’s
DevOps transformation was complete, in progress, scheduled to kick off, or not a priority at all.

How testers spend their time evolves significantly as DevOps maturity increases
Across all DevOps maturity levels, survey respondents report spending most of their time, on average,
performing these five tasks:
• Implementing tests
• Designing tests
• Planning tests
• Testing strategy
• Documenting issues

When we broke down the results by DevOps maturity, we found that in organizations where DevOps
transformation is complete or in progress, testing strategy and updating/maintaining automated tests
rank in the top 5 places testers spend their time. The only place testing strategy is first on the list is in
organizations where DevOps transformation is complete. It is not listed in the top five in organizations
where DevOps isn’t a priority.

In organizations where DevOps transformation hasn’t started, implementation is rated number one and
documentation (documenting issues found and documenting manual tests) replaces updating/maintaining
automated tests in the top 5.

How are testers spending their time?

DevOps Initiative Not Started DevOps Initiative In Progress DevOps Initiative Complete

1. Implementing tests 1. Implementing tests 1. Testing strategy


2. Designing tests 2. Planning 2. Implementing tests
3. Documenting manual tests 3. Designing tests 3. Planning tests
4. Documenting issues 4. Testing strategy 4. Designing tests
5. Planning tests 5. Updating/maintaining 5. Updating/maintaining
automated tests automated tests

8
2019 Tricentis Survey Report

How do testing pain points evolve with DevOps maturity?


As the role of tester evolves, it’s natural that the changes will bring some level of discomfort. Some of the
biggest testing pains reported by survey respondents are shown below. In addition to test environment
management, test maintenance, and test data management, testers cited other pain points, including lack of
testing resources, lack of disciplined testing, and too much manual testing.

Biggest testing pains

15% Test environment management

26%
Test maintenance

18% Test data management

25% Other

We wanted to know how testers’ pains change with level of automation and DevOps transformation, i.e., the
“rewards of test automation maturity.” We found some big differences in the most common pain points at
differing levels of both DevOps maturity and test automation.

Survey results indicate that almost every organization, at every level of DevOps maturity, struggles with test
maintenance. For those organizations that haven’t started DevOps transformation or that are planning to
scale test automation, it’s critical to get this under control, because it’s only going to become more difficult
as the transformation progresses. Test maintenance is the first problem teams adopting test automation
are likely to encounter.

We also found that the higher the DevOps maturity, the more pain test maintenance causes, and the
lower the DevOps maturity, the more pain test orchestration, or simply running test cases, causes. We can
extrapolate that because these teams are likely just getting started with test automation and automating
a narrower range of scenarios, they are not yet struggling with maintenance, but are still ironing out the
processes associated with orchestration.

“Our biggest pain point is the mish mash of different team


methodologies—one is agile, one waterfall, one uses continuous—
the resulting clashes of integrated systems is a bit of a nightmare.”

9
2019 Tricentis Survey Report

When we broke down pain points by level of automation, we observed similar trends. Nearly a third (29%) of
organizations with lower levels of automation (0% to 10% of test effort) say test orchestration is a pain point
but only 9% of those with higher levels of automation (40%+ of test effort) struggle with test orchestration.
Organizations that are further along in their automation journey tend to cite test maintenance, test data
management, and test environment management as the most significant pain points.

How testing pains change with DevOps maturity

DevOps DevOps DevOps is a priority,


DevOps is not a
transformation is transformation is in but we have not
priority
complete progress started
Test maintenance 27% 31% 19% 14%
Test data management 27% 20% 6% 20%
Test environment
20% 31% 16% 23%
management
Test orchestration 7% 6% 19% 14%

How testing pains change with level of automation

Level of Automation
More than
0% 1% to 10% 11% to 20% 21% to 30% 31% to 40% 41% to 50%
50%
Test reporting 21% 8% 7% 3% 0% 0% 4%
Test maintenance 14% 17% 21% 32% 43% 18% 36%
Test data
14% 13% 25% 6% 29% 18% 32%
management
Test environment
14% 27% 25% 32% 14% 45% 20%
management
Test
14% 15% 11% 13% 0% 9% 0%
orchestration

In organizations with lower levels of automation (0% to 10% of test effort), test reporting is a common
struggle. In this category, 29% of respondents say test reporting is a pain point while only 4% of those with
higher-level automation (50%+ of test effort) agree. This suggests that as organizations kick off or scale a
test automation initiative, they struggle to adjust their reporting to the new processes. Reporting can be
especially difficult to iron out in organizations using multiple open source test automation tools, which
requires them to analyze results across multiple tools.

10
2019 Tricentis Survey Report

Where do testers make the most tradeoffs?


Testers frequently find that they must make tradeoffs in order to accomplish all that they are required to.
We wanted to know what is driving these tradeoffs and whether testers are experiencing other problems
with the testing process.

We found that more than half of testers (55%) are forced to make tradeoffs because of lack of time while
16% report tradeoffs occur because they don’t have the right tools and capabilities.

Top reasons testers make tradeoffs

16% Lack of time

Insufficient budget
9%
55% Insufficient tester skill levels
9%
Insufficient tool availability and capabilities

Lack of time has implications for testers beyond making tradeoffs. Almost a third of respondents (32%) cite
not having enough time to test as the biggest problem with their organization’s testing process across the
board. Unclear stakeholder expectations (21%) also made the list of testing process problems testers have
to deal with.

Common problems with the testing process

Unclear stakeholder expectations 21%

Insufficient time for testing 32%

Insufficient tools for testing 9%

Insufficient skills for testing 15%

Lack of developer collaboration and assistance 9%

0 5 10 15 20 25 30 35

Percentage of Respondents

11
2019 Tricentis Survey Report

Lifecycle Models and Testing Practices

With all of the emerging lifecycle models and testing practices over the past several years, we were curious
whether testing practices or lifecycle model impacts job satisfaction.

The survey didn’t reveal a relationship between job satisfaction and testing practices, such as AI-augmented
testing, BDD, DevOps, etc., but the results do show that a majority of testers are generally more satisfied than
dissatisfied with their jobs.

We did, however, see a correlation between the level of job satisfaction and lifecycle model. Those testers
working in a primarily waterfall environment reported lower levels of job satisfaction than testers working in
primarily agile environments. This could be due to several factors, including the increased collaboration and
communication on agile teams, the opportunity to learn new technical skills, and being able to find and fix
issues as they occur rather than waiting until production to discover an error or bug.

Relationship between job satisfaction and lifecycle model

Waterfall Agile Hybrid Other

Very satisfied 0% 16% 13% 0%

Satisfied 14% 38% 31% 2%

Neutral 14% 12% 21% 4%

Dissatisfied 14% 12% 8% 5%

Very dissatisfied 14% 2% 7% 0%

“Every tester goes through difficulties with changes in


requirements, deadlines, and expectations regarding
automating tests, reporting test results, etc. I feel it is part of
the job in agile teams. We learn to adjust and we get things
done. That doesn‘t make me dissatisfied with my role.”

12
2019 Tricentis Survey Report

How Agile Adoption Has Affected


Software Testing

Over the past 15 years we’ve seen agile adoption explode within organizations of all sizes and industries.
Before DevOps came along, software testers were concerned that agile was going to put them all out of a job.
We wanted to know what effect agile really has had on testers, so we asked respondents to rate how agile
impacted various areas of testing in their organization. Some of the more significant findings show that as a
result of agile implementation within the organizations surveyed:
• Tester value increased
• Testing scope became broader
• Testing became more thorough
• Job satisfaction increased

“We do not have a manual tester role. We have


BAs that cover some manual testing, users that
are able to cover in-depth testing, and a QA
and automation team that helps train and guide
testing efforts and covers automated testing.”

The Changing Face of Testing

Industry notables have been prematurely announcing the death of software testing for years [5], and yet
here we are. As time goes on and development methodologies wax and wane in popularity, there is only one
thing we can predict with certainty: change is inevitable.

While the widespread adoption of agile and DevOps prompted their share of hand wringing, the tester role
shows no sign that it will become obsolete. Though it will, without a doubt, be different.

13
2019 Tricentis Survey Report

Survey respondents identified several potential threats to their jobs, but expectation of programming/
scripting skills (33%) and testing being deprioritized (47%) are by far the biggest perceived threats. When we
look at the perceived threats by salary range, we found that half of respondents who earn less than $50K per
year are most threatened by testing being deprioritized while only 25% of those earning more than $150K
per year feel the same.

Perceived threats to testers’ jobs

DevOps DevOps DevOps is a


DevOps is not a
All respondents transformation transformation priority, but we
priority
is complete is in progress have not started
AI-based testing
5% 0% 7% 3% 3%
tools
SDET and
9% 0% 9% 13% 9%
developer testing
Robot process
2% 0% 2% 0% 3%
automation (RPA)
Expectation of
programming/ 33% 13% 37% 44% 20%
scripting skills
Testing being de-
47% 73% 39% 38% 63%
prioritized
Other 6% 13% 7% 3% 3%

Given that testing being deprioritized and lacking programming skills are the top two threats to testing
jobs according to survey respondents, it makes a lot of sense that SDET or test automation roles (39%) and
DevOps roles (22%) are seen as the next great opportunities for testers. Preparing early for the future of
testing is an excellent risk mitigation strategy and a smart career move.

Next great opportunity for software testers

DevOps roles 22%


RPA and enterprise automation 7%
BI/data warehouse testing 3%
Business analyst roles 16%
SDET or test automation roles 39%
Development roles 7%

0 5 10 15 20 25 30 35 40
Percentage of Respondents

14
2019 Tricentis Survey Report

Manager satisfaction improves with increased levels of test automation and


DevOps maturity
Survey findings show that as DevOps maturity and level of test automation increases, testing managers’
satisfaction with their team’s work also increases. This is particularly evident in organizations where DevOps
transformation is complete and among those that automate greater than 50% of their testing effort.

In organizations where DevOps transformation is complete, 66% of managers are very satisfied or satisfied
with their team’s work compared with 23% of managers in organizations where DevOps is not a priority.

Organizations that don’t automate testing report only 14% of managers are very satisfied or satisfied with
their team’s work. In those organizations with 50%+ automation, 64% of managers are very satisfied or
satisfied with the work their teams do.

How manager satisfaction changes with DevOps maturity and level of automation

Very
DevOps maturity Very satisfied Satisfied Neutral Dissatisfied
dissatisfied

DevOps transformation is
13% 53% 0% 7% 7%
complete

DevOps transformation is in
19% 33% 14% 7% 0%
progress

DevOps is a priority, but we


19% 28% 19% 9% 0%
have not started

DevOps is not a priority 6% 17% 23% 9% 6%

Very
Level of test automation Very satisfied Satisfied Neutral Dissatisfied
dissatisfied

0% 0% 14% 36% 7% 14%

1% to 10% 17% 27% 17% 6% 2%

11% to 20% 11% 32% 14% 18% 0%

21% to 30% 26% 23% 10% 10% 0%

31% to 40% 21% 36% 7% 0% 0%

41% to 50% 0% 45% 27% 0% 0%

More than 50% 20% 44% 12% 4% 0%

15
2019 Tricentis Survey Report

We know that manager satisfaction with their team’s work increases with DevOps maturity and level of
automation, but how do these factors affect the testers themselves?

Testers in organizations that don’t prioritize DevOps report higher job satisfaction and lower job
dissatisfaction than testers in organizations that have completed DevOps transformation. Job satisfaction is
lowest among organizations that are currently undergoing, but haven’t completed, DevOps transformation.

It’s likely this increase in job satisfaction for testers moving


to DevOps is a result of the transformation process itself.
If an organization is in the process of transitioning to DevOps and is not prioritizing testing, it should not
come as a surprise if tester job satisfaction decreases. If testers aren’t yet enabled to test at the speed of
DevOps releases, they’ll be perceived as the bottleneck. But if testers are enabled to modernize, integrate
with agile development, and scale test automation, they’re likely to be more satisfied. They will be equipped
to meet expectations of their counterparts and stakeholders, and they will be learning new things. In any
job or industry, innovating and modernizing is likely to be much more satisfying than continually being
viewed as a bottleneck.

How testers’ job satisfaction is affected by DevOps maturity and level of automation

Very
DevOps maturity Very satisfied Satisfied Neutral Dissatisfied
dissatisfied
DevOps transformation is
13% 33% 7% 13% 7%
complete
DevOps transformation is in
16% 28% 21% 11% 6%
progress
DevOps is a priority, but we
13% 38% 3% 19% 0%
have not started
DevOps is not a priority 9% 46% 20% 3% 6%

Very
Level of test automation Very satisfied Satisfied Neutral Dissatisfied
dissatisfied
0% 0% 29% 36% 14% 0%
1% to 10% 8% 40% 13% 6% 15%
11% to 20% 21% 25% 18% 18% 0%
21% to 30% 13% 39% 23% 10% 0%
31% to 40% 21% 36% 7% 7% 0%
41% to 50% 9% 36% 9% 18% 9%

More than 50% 20% 28% 12% 12% 0%

16
2019 Tricentis Survey Report

Conclusion

The 2019 survey results indicate


that testing is a critical element in
successfully transitioning to DevOps.
We found that testers in organizations
where DevOps transformation is
complete feel that their testing effort
was integral to the process, and
that their organizations were more likely to prioritize testing. We also discovered that automation levels
increase with DevOps maturity, but that it is more difficult to scale than previously anticipated. In fast-paced
DevOps environments, the basics of software quality still apply. So we can expect that testers will continue
to be integral to the success of any organization that develops or customizes software.

We took a look at where testers are spending their time and whether DevOps maturity affected this. We
discovered that regardless of where an organization is in its DevOps transformation, testers spend the
most time implementing, designing, and planning tests; working on testing strategy; and documenting
issues. However, we did find that in organizations with greater DevOps maturity testers spend more time
on strategy and maintaining automated tests than their peers in organizations that haven’t implemented
DevOps at all. These testers spend the bulk of their time implementing tests and documenting issues and
manual tests.

Again, we find consensus across every level of DevOps maturity when it comes to the pain of test
maintenance, though testers in organizations with high DevOps maturity rate maintenance as their top
pain point. Test orchestration was more painful for those in low DevOps and low test automation adoption
companies.

Testers across the board feel threatened by the prospect of testing being deprioritized and by having to
learn more technical/programming skills. While it’s unlikely that the testing role will become obsolete, there
is no doubt that it is evolving, and testers will need to take steps now to keep up with the changing face of
testing. Likewise, technology leaders who are planning or in the midst of a DevOps transformation must
take steps to ensure their testing teams are enabled to transform with the rest of the business.

17
2019 Tricentis Survey Report

The survey found that manager satisfaction improves with increased levels of test automation and
DevOps maturity, and that in organizations that have completed DevOps transformation, managers are
three times more likely to be satisfied with their team’s work. Findings also show a four-fold increase in
manager satisfaction between organizations with 0% test automation (14%) and those with more than 50%
automation (64%). These findings suggest that when testers are set up for success and prioritized alongside
other key functions, they are better equipped to meet expectations, and their organizations are more likely
to succeed with their DevOps initiatives.

“I really like the work my team is


doing and the growth I‘m seeing as we
implement BDD and DevOps practices.”

REFERENCES
[1] 2019 Global Developer Survey: DevSecOps. Gitlab.

[2] 2017-2018 World Quality Report. Sogeti.

[3] 2019 World Quality Report. Sogeti.

[4] The Evolution of Test Automation. Tricentis. 2018.

[5] On the Death of Testing … and Wildebeests, Wolfgang Platz. 2019.

18
2019 Tricentis Survey Report

Appendix 1:
Regional Comparison
of Key Survey Results

Testers from around the world contributed to the 2019 survey Expectations vs. Reality: The Role of Testing
in a DevOps Transformation. While the results show many global similarities in how the role of testing is
evolving to meet the demands of DevOps and increased automation, some regional differences emerged,
which are highlighted here.

The regions are broken down into Europe/Middle East/Africa (EMEA), the Americas, and Asia Pacific (APAC).

The APAC region 50%+ Automation 0% Automation


automates
significantly APAC 27% APAC 3%
more testing EMEA 16% EMEA 7%
than other
regions AMERICAS 15% AMERICAS 8%

The APAC region has


the lowest percentage Respondents
reporting no DevOps
of organizations who 12% 25% transformation has
haven’t prioritized APAC EMEA occurred because it’s
DevOps transformation; not a priority

EMEA has the highest

19
2019 Tricentis Survey Report

In the EMEA region, testing isn’t considered as


crucial to DevOps transformation
Respondents reporting testing is “extremely important” to DevOps transformation:

APAC 77%

AMERICAS 72%

EMEA 47%

Businesses in the APAC region Testers in the APAC region get


prioritize testing as part of the more respect compared with
DevOps transformation process testers in other regions
more than other regions
Respondents who report that testing is “extremely
Respondents reporting that their organization has well respected”:
prioritized testing as part of DevOps transformation:

45%
71%
50% 49% 26% 26%

APAC EMEA AMERICAS APAC EMEA AMERICAS

Testers in the APAC


Percent of respondents who are “very satisfied” with their roles:
region report the
highest levels of job
satisfaction

24%
APAC 13% 9%
AMERICAS EMEA

20
2019 Tricentis Survey Report

Percent of “very satisfied” managers:


Managers in the
EMEA region are EMEA 6%
less satisfied
12%
with the testers APAC

they manage AMERICAS 16%

How do salaries compare across regions?

TESTERS IN THE AMERICAS EMEA IS THE LOWEST


EARN THE MOST PAID REGION APAC

6% make $25K or less


6% make $25K or less 10% make $25K or less

33% make $100K or more


42% make $100K or more 9% make $100K or more

Salaries are USD equivalent

21

You might also like