Dev Ops Maturity Models
Dev Ops Maturity Models
net/publication/336114851
Article in International Journal of Recent Technology and Engineering (IJRTE) · September 2019
DOI: 10.35940/ijrte.C6888.098319
CITATIONS READS
34 9,131
4 authors, including:
Mamdouh Alenezi
Prince Sultan University
156 PUBLICATIONS 2,479 CITATIONS
SEE PROFILE
All content following this page was uploaded by Mamdouh Alenezi on 28 September 2019.
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4854 & Sciences Publication
A Research on DevOps Maturity Models
DevOps concepts and adoption that should follow this platforms that do not run on IBM software. Another limitation
research work. is that it does not provide a clear justification on the
investment strategy for achieving DevOps maturity. IBM
2.1 IBM DevOps Maturity Model
DevOps maturity model has 4 levels, as follows, See Fig. 1:
Bahrs, P form IBM [6], provided a thorough analysis on the Level-1 is “Practiced”: At this level, the enterprise
adoption of IBM DevOps approach for promoting continuous standards are not defined, inconsistent automation, and teams
delivery of software. The author identified four dimensions in may perform some activities associated with the practice
adopting or implementing continuous software growth within inconsistently.
an organization. These dimensions include Planning and Level-2 is “Consistent”: The enterprise standards at this
measuring, Developing and testing, Releasing and deploying, level are defined, automation follows the standards, and teams
and Monitoring and optimizing. The IBM DevOps maturity perform activities associated with the practice according to
model is a practice-based and reflects a wider context within the standards.
the adoption framework of an organization. It focuses on Level-3 is “Reliable”: At this level, enterprise’s standards
defining the best practices to be applied in the adoption of are being followed, an exist mechanisms to assist adoption, a
new software solutions iteratively. mentor team is available to assist in adopting the best
A well-articulated approach for assessing current DevOps practices.
practices within an organization is also provided in [6]. It also Level-4 is “Scaled”: At this level, institutionalized
helps in defining a clear roadmap for DevOps practices are defined for the adoption across the enterprise,
implementation. Furthermore, the mentioned research work matured core team is formulated, and feedback process is
provided its readers with a high-quality approach for established for the standards improvement.
measuring the improvement made by an organization in
implementing the IBM DevOps approach. Most importantly,
this DevOps maturity provides a clear set of steps for
preparing, piloting and releasing system improvements within
an organization.
It is important to note that his model does not specify the
applicability of the IBM DevOps approach in other software
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4855 & Sciences Publication
International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8 Issue-3, September 2019
Level-1 is “Initial”: At this level, ad-hoc communication metrics, automation metrics, and governance metrics, quality
with no clear process, no automation implemented, metrics exist for improvement and measurement.
uncontrolled governance/process where the outcome of any Level-5 is “Optimized”: At this level, constructive
service is not predictable, and no quality standards exist. The communication environment, tools and processes are
whole activities are done based on the process owner adopted, smart automation to maximize throughput,
objectives. optimized governance self-adaption, and continuous quality
Level-2 is “Managed”: At this level, communications are improvement.
controlled but not shared between teams, documented
automation process but not executed, executed governance
but not standardized, and ad-hoc quality management is in
place.
Level-3 is “Defined”: At this level, the communication,
automation, and governance are standardized. Quality
standard exists.
Level-4 is “Measured”: At this level, the communication
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4856 & Sciences Publication
A Research on DevOps Maturity Models
Level-5 is “Top level”: One collaborative team with full
knowledge sharing. There is a single process that cover the
entire solution lifecycle and organization strategy. Setup for
all environments are automated from one single repository.
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4857 & Sciences Publication
International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8 Issue-3, September 2019
2.5 Bucena DevOps Maturity Model adopted. The delivery process is automated, and integrated.
Bucena and Kirikova [12], have developed a DevOps The team is organized around projects, the communication
maturity model based on CMM approach, which consist of between teams are rapid, clear project requirements, active
five maturity levels. Each of these levels has four dimensions collaboration, and identified culture traits.
which are technology, process, people and culture. It is Level-4 is “Managed”: The environments are managed
defined as follows, See Fig. 5: effectively, smoked tests shared with operation team,
Level-1 is "Initial": The environments, tests, data production deployment is automated. The process delivers
migration, and deployment are performed manually. The frequently, visible and predictable. The team organized
delivery process and project management are inconsistent, around products, frequent collaboration and communication,
ad-hoc approaches for learning. Communication is restricted, clear product requirements, and culture viewed as asset to be
and lack of awareness as how the culture impacts day to day managed.
business. Level-5 is “Optimized”:
Level-2 is “Repeatable”: The environments configurations The environments fully automated, continuous work on
are externalized and versioned, the delivery process is process improvement for better visibility and faster feedback,
scheduled, project and requirement are managed, requirement continuous delivery process, and the collaboration between
is based on testing, development documents are up-to-date, operation and development teams to manage risks and reduce
scrum development, and managed processes but not cycle time. Teams are organized around KPIs. Fig.5
standardized, the team organized around deliveries, In the illustrates a sample of this maturity model.
culture dimension, the communication among internal team
are rapid.
Level-3 is “Defined”: Environments virtualization is
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4858 & Sciences Publication
A Research on DevOps Maturity Models
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4859 & Sciences Publication
International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8 Issue-3, September 2019
3.1 Maturity Models’ Levels Names enhanced or measured, but they all reflect the same meaning
From our investigation of the DevOps maturity models, we of having a managed process:
have noticed that, see Table 2: Level 1: A level for the starting point for use of the new
There are two similar models which are: Mohamed’s process (Initial).
model, and Inbar’s model because both of them comply with Level 2: A level where the process is at least documented
CMMI. sufficiently (Managed).
There are three similarities in the levels’ names between Level 3: A level where the process is defined as a standard
Mohamed’s model, Inbar’s model, and Bucena’s model process (Defined).
which are level 1,3, and 5. Level 4: A level where the process is managed in
The level names are different from one maturity model to accordance metrics (Measured).
another but their meanings, may have some similarities and Level 5: A level where the process managed efficiently
differences. For instance, the first level (initialization) for (Optimized)
Bahrs’s model is named “Practiced” while in Menzel’s model
it is named “Basic”, and the others name it “Initial”.
Eficode and Feijter models used numbers to name the
maturity levels
The fourth level has different names, e.g. scaled, managed,
Table 2. Maturity Models’ Levels
Maturity Level 1 Level 2 Level 3 Level 4 Level 5
model
Bahrs’s model Practiced Consistent Reliable Scaled NA
Mohamed’sm
Initial Managed Defined Measured Optimized
odel
Menzel’s
Basic Emerging Co-ordinated Enhanced Top level
model
Inbar model Initial Managed Defined Measured Optimized
Bucena model Initial Repeatable Defined Managed Optimized
Eficode’s
1 2 3 4 NA
model
Feijter model Differ than other models with 10 maturity levels
One odd model has 10 maturity models that is Feijter
3.2 Maturity Models’ Number of Levels, Publication Year, model.
Number of Dimensions, and Application Regarding the maturity models’ publishing year, note that
From Table 3, we noticed that the maturity models have all the models are relatively new and published in the period
different number of levels: of 2013-2018 which justifies the scarce of published work
Two maturity models have four levels, Bahrs’s and related to DevOps, which we expect to increase the coming
Eficode’s models. few years. Furthermore, regarding the number of dimensions,
Four maturity models have five levels, which are: note that all models have 3-5 possible dimensions, as
Mohamed’s model, Inbar’s model, Menzel’s model and discussed in the next section.
Bucena’s model.
Table 3. Maturity Models’ Number of levels, Year
Number Pub. Number of Organizations
Maturity model
of levels Year dimensions Validated the model
Bahrs’s model 4 2013 4 1 (IBM)
Mohamed’s model 5 2015 4 -
Menzel’s model 5 2015 3 -
Inbar’s model 5 2013 3 1 (Hewlett Packard)
Bucena’s model 5 2017 4 1 Anonymous SMEs
Eficode’s model 4 2015 5
Feijter Model 10 2018 3 1 (Centric)
G. Menzel and A. Macaulay [4] has tool dimension, Bucena
3.3 Maturity Models’ Dimensions
[6] has a technology dimension and Eficode [8] has a
Different DevOps maturity models have different environment and release dimension that contains automation
dimensions, see Table 4: and tools as sub dimension.
Four maturity models have Process dimension which are:
Mohamed’s model, Menzel’s model, Inbar’s model, and
Bucena’s model.
In the second dimension (dimension B), Mohamed, S [3],
and Inbar et al. [5] models have automation dimension, where
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4860 & Sciences Publication
A Research on DevOps Maturity Models
Most of the maturity models have dimension for team Bahrs, P [2] model dimensions revolve on the process
collaboration, as in Mohamed, S [3] and Inbar et al. [5]. G. automation
Menzel and A. Macaulay [4], Bucena [6] call it people and One model has fifth dimension related to visibility and
differentiate it from the culture dimension, Eficode and reporting
Feijter models call it culture dimension.
Table 4 Maturity Models’ Dimensions
Dimension Dimension Dimension Dimension Dimension
Maturity model
A B C D E
Bahrs’s model Plan Develop Release Monitor -
Mohamed’s Collaboration Automation Process / Quality -
model Governance
Menzel’s model People Tool Process - -
Inbar’s model Collaboration Automation Process - -
Bucena’s model People Technology Process Culture -
Eficode’s model Organization Environments Builds and Quality Visibility &
and culture and release continuous assurance reporting
integration
Culture & Product, Process Foundation - -
Feijter Model
Collaboration & Quality
V. REFERENCES
IV. CONCLUSION AND FUTURE WORK 1. P. Rodríguez et al., “Continuous deployment of software
To conclude, it is clear from the conducted comparison that intensive products and services: A systematic mapping
study,” J. Syst. Softw., vol. 123, pp. 263–291, Jan. 2017.
most of the maturity models have 5 levels either following
2. F. M. A. Erich, C. Amrit, and M. Daneva, “A qualitative
CMM or CMMI, others have 4 and one exceptional model has study of DevOps usage in practice,” J. Softw. Evol.
10. Regarding the experiment application two of the models Process, vol. 29, no. 6, p. e1885, Jun. 2017.
are applied and validated by the institute that already 3. J. Sharp and J. Babb, “Is Information Systems Late to the
developed the mode while other to models are validated by Party? The Current State of DevOps Research in the
applying the model in one independent organization. This Association for Information Systems eLibrary,” in
AMCIS 2018 Proceedings, 2018.
means that the usage of the DevOps maturity models is still
4. J. Becker, R. Knackstedt, and J. Pöppelbuß, “Developing
very limited and is not yet used by variety of organizations Maturity Models for IT Management,” Bus. Inf. Syst.
and this would make the validity of such models questionable. Eng., vol. 1, no. 3, pp. 213–222, Jun. 2009.
Accordingly, more research and empirical work is vitally 5. D. Proença and J. Borbinha, “Maturity Models for
needed to practice and validate the proposed DevOps Information Systems - A State of the Art,” Procedia
maturity models. Comput. Sci., vol. 100, pp. 1042–1049, Jan. 2016.
6. P. Bahrs, “Adopting the IBM DevOps approach for
It is also noted that there are large similarities in the
continuous software delivery: Adoption paths and the
measured dimensions in most of models expect Bucena’s and DevOps maturity model,” 2013.
Feijter’s models. Both models assist the culture dimension 7. I. Bucena and M. Kirikova, “Simplifying the devops
and both of them have validated their model in one adoption process,” CEUR Workshop Proc., vol. 1898,
organization from the industry, i.e. an independent 2017.
organization that they do not work for. Moreover, we found 8. R. de Feijter, S. Overbeek, R. van Vliet, E. Jagroep, and S.
Brinkkemper, “DevOps Competences and Maturity for
that Bucena’s model is holistic and covers all of DevOps
Software Producing Organizations,” in Enterprise,
dimensions while Feijter’s model is dedicated for SPO Business-Process and Information Systems Modeling,
organizations. Hence, we believe that Bucena’s and Eficode’s vol. 318, Springer, 2018, pp. 244–259.
models are comprehensive and promising models to build on 9. Eficode Oy, “DevOps Quick Guides,” Helsinki Finland,
them and conduct future assessment to assess DevOps 2015.
maturity. 10. S. Inbar, S., Sayers, Y., Pearl, G., Schitzer, E., Shufer, I.,
Kogan, O., & Ravi, “DevOps and OpsDev: How Maturity
Another observation concerning the application of the
Model Works,” Hewlett Packard Enterprise, 2013.
various DevOps maturity models, is that none of the 11. G. Menzel and A. Macaulay, “DevOps - The Future of
researchers have documented the adopted assessment method Application Lifecycle Automation,” Capgemini.Com, p.
in an academic publication. All of them documented the 24, 2015.
model and its applications then discussed the findings and 12. S. Mohamed, “DevOps shifting software engineering
results. We believe that this is not enough. Developed strategy-value based perspective,” Int. J. Comput. Eng.,
vol. 17, no. 2, pp. 51–57, 2015.
assessment method should also be published to be used or 13. S. I. Mohamed, “DevOps Shifting Software Engineering
enhanced by other researchers. The next step is to use one of Strategy Value Based Perspective,” IOSR J. Comput.
the recommended models to assess the maturity of Saudi Eng. Ver. IV, 2015.
organization in adopting DevOps via an empirical study.
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4861 & Sciences Publication
International Journal of Recent Technology and Engineering (IJRTE)
ISSN: 2277-3878, Volume-8 Issue-3, September 2019
AUTHORS PROFILE
Mohammad Zarour
Dr. Zarour holds a Ph.D. in Software
Engineering (2009) from University of Quebec and
master degree in Computer Science (1998) from
University of Jordan. He is currently a faculty
member in college of computer and information
sciences (CCIS) at Prince Sultan University
Riyadh, Saudi Arabia. He has more than 12 years
of teaching experience in university and academic
environment and also has several years of industry
experience in information systems development and project management.
His research interests include software process assessment and
improvement, software quality, cost estimation, and web technologies. He
has many peer-reviewed publications.
Norah Alhammad
Mrs. Norah has a master degree in software engineering from Prince
Sultan University Norah has a Master degree in Software Engineering from
Prince Sultan University. Her work focus in software development and
DevOps. Currently, she is an Information Technology Project Manager in
Semi Government Industry.
Mamdouh Alenezi
Dr. Alenezi is currently the Dean of
Educational Services and the Chief
Information & Technology Officer (CITO) at
Prince Sultan University. Dr. Alenezi
received his MS and Ph.D. degrees from
DePaul University and North Dakota State
University in 2011 and 2014, respectively. He
has extensive experience in data mining and
machine learning where he applied several
data mining techniques to solve several
Software Engineering problems. He conducted several research areas and
development of predictive models using machine learning to predict
fault-prone classes, comprehend source code, and predict the appropriate
developer to be assigned to a new bug.
Khalid Al-Sarayreh
Dr. Al-Sarayreh is an associate professor of
Software Engineering at the Prince Sultan
University in Saudi Arabia. He has a Ph.D.
degree in Software Engineering from the
University of Québec in Canada. He also has a
doctoral degree in Computer Information
Systems, MSc in Computer Engineering
(Embedded Systems) and BS degree in
Computer Science from Jordanian Universities. He served during 25 years in
both national and international institutions. With over 70 publications, his
research interests include Software Quality Engineering, Software Quality
Assurance using international standards, Software Requirements
(Functional and Nonfunctional requirements), Software Measurement,
Software Reuse, Software Engineering Standards (ECSS, IEEE, and ISO).
Published By:
Retrieval Number C6888098319/2019©BEIESP Blue Eyes Intelligence Engineering
DOI: 10.35940/ijrte.C6888.098319 4862 & Sciences Publication
View publication stats