Proceedings of I-KNOW ’09 and I-SEMANTICS ’09

2-4 September 2009, Graz, Austria 51

Understanding Maturity Models

Results of a Structured Content Analysis
Michael Kohlegger, Ronald Maier, Stefan Thalmann
(University of Innsbruck, School of Management, Information Systems, Austria
[firstname.lastname]@uibk.ac.at)

Abstract: Maturity models are popular instruments used, e.g., to rate capabilities of maturing
elements and select appropriate actions to take the elements to a higher level of maturity. Their
application areas are wide spread and range from cognitive science to business applications and
engineering. Although there are many maturity models reported in scientific and non-scientific
literature, the act of how to develop a maturity model is for the most part unexplored. Many
maturity models simply – and vaguely – build on their, often well-known, predecessors without
critical discourse about how appropriate the assumptions are that form the basis of these
models. This research sheds some light on the construction of maturity models by analysing 16
representative maturity models with the help of a structured content analysis. The results are
transformed into a set of questions which can be used for the (re-)creation of maturity models
and are answered with the help of the case example of a knowledge maturity model.
Furthermore, a definition of the term maturity model is developed from the study’s results.

Keywords: capability, maturity model, knowledge maturing, structured content analysis

Categories: H.1.0, H.3.1, M.9

1 Maturity modelling and knowledge maturing

Maturing has been used as an analytic, explanatory or normative concept in several
domains, the most well-known of which is software engineering, e.g., Nolan’s stage
theory [Nolan 1973], the capability maturity model integration (CMMI,
http://www.sei.cmu.edu/cmmi/) or the Software Process Improvement and Capability
Determination model (SPICE, http://www.isospice.com/). The identification of 74
different maturity models from domains within the spectrum of (business)
information systems and computer science shows the great variety and widespread
use of maturity models available in the literature. However, this list only provides the
tip of an iceberg due to the fact that there are many more maturity models in other
domains, e.g., biology , sociology or psychology [Greenberg, et al. 1974, 331pp].
An analysis of a sample of maturity models has shown that many models differ
with respect to their characteristics. But at the same time also several similarities
within these models have been found which in part can be explained by the fact that
many authors of maturity models simply build on their predecessors without much
thinking about the appropriateness of their design decisions. This suggests that the
concept of maturity modelling should be reflected and reassessed.
As a deeper understanding of maturing and maturity models is necessary for the
creation of a new or revised maturity model, goals of this paper are to give an
overview of the different conceptions of maturing and fundamental principles of
maturity models and show the applicability of the results with the help of the case
52 M. Kohlegger, R. Maier, S. Thalmann: Understanding ...

example of a knowledge maturity model. The paper presents the results of an in-depth
analysis of 16 maturity models for their individual characteristics in order to gain
information about the nature of maturing and maturity modelling. The results are then
used to inform future development of maturity models in the domains of (business)
information systems and computer science with a special emphasis on the field of
knowledge management, i.e. knowledge maturing. Aspects of how these findings
could be used to inform the creation of new or the revision of existing maturity
models are discussed with the help of the example of a knowledge maturity model.
Section 2 gives an overview of the procedure taken for the analysis of maturity
models. Section 3 presents and discusses the results of the structured content analysis
of 16 maturity models. Beyond that, section 4 proposes a definition for the term
maturity model, before section 5 concludes this paper.

2 A study design for the analysis of maturity models

The investigation into the nature of maturity and the act of maturity modelling was
performed with the help of the structured content analysis method. This section
presents the applied study design for the investigation of 16 maturity models.
Analysis in the scientific world is often distinguished into qualitative and
quantitative approaches. Quantitative research strategies are based on the model of the
natural sciences and therefore facilitate the testing of hypotheses. Qualitative research
strategies are based on the model of the social sciences and follow an inductive
approach of research. This approach could also be described as an understanding
approach which is often used to build hypotheses. [Schambach-Hardtke 2005, 18].
The aim of the current research is to explore and analyze existing models and to
investigate characteristics and not to test hypotheses which would require a prior
investigation. This procedure of identifying characteristics demands a qualitative
approach. We decided to rely on textual descriptions of maturity models as sources
for our investigation rather than empirical evidence in order to be able to cover more
models and compare them. A suitable method focusing on the analysis of contents
and the identification of criteria is the structured content analysis [Mayring 2008,
82pp]; its general procedure is depicted in [Figure 1].
 M. Kohlegger, R. Maier, S. Thalmann: Understanding ... 53

Figure 1: Process flow of a structured content analysis [Mayring 2008, 84]

Structured content analysis starts with the definition of the analysis items (1), which
are those items that provide descriptions about the model itself. The smallest analysis
item can be a single word while the potentially largest item is the entire model
description.
In the next step, information associated with an analysis item is extracted from
the model context and put into another structure determined by several structuring
dimensions (2) and its underlying category system (3). In this paper, the structure was
determined by the analysis question “Which characteristic patterns can be observed
by comparing maturity models?” that comprises four sub-questions:
- What does the term maturing mean? Aim is to find the underlying concept of
maturing employed by the model.
- Which element is maturing? Aim is to classify the subject of maturing into person,
object or social system and some more detailed categories.
- What are the main features of a maturity model? Aim is to find a construction
plan for maturity models. Because the authors were not aware of any similar
studies, no codes were available, hence an inductive approach was needed to
develop these.
54 M. Kohlegger, R. Maier, S. Thalmann: Understanding ...

- How is the model used and supported? Aims to answer the question why some
maturity models are applied in practice while others are not. For example for
[Krcmar 2005], formal certification is a critical factor.
After the preparation phase which ends with a verbal definition of the derived
categories (4), the material is worked through (5 and 6) for a first time. As new
categories my arise during this first run, [Mayring 2008, 83] proposes an iterative
approach that allows rework of the category system (7). This inductive process is
typical for qualitative techniques [Lamnek 2005, 91].
The material was worked through in three phases [see Figure 2] while the number
of codes first rapidly increased (phase A), then slowly got more and more stable
(phase B) to be finally condensed to a concluding level (phase C). The revision points
between the phases were set at 13% (2 models), 50% (8 models) and 100% (16
models) of the models worked through.

Figure 2: The three steps of the coding process. based on [Hädrich 2008, 222pp]

As suggested by [Dey 1993, 120], the analysed models were selected randomly in
order to avoid a coding bias which might have arisen from how the models were
selected and would have strained the development of the category system.
An Internet search was carried out in order to get an approximation of the basic
population of maturity models as basis to draw a sample resulting in a list of 76
maturity models, further subdivided into three categories:
- maturing of persons (8 models found)
- maturing of objects (22 models found)
- maturing of social systems (46 models found)
The classification into these three categories is in line with knowledge as
maturing element that can either be connected to a person, an object or a social
system [Maier 2007, 198pp]. In order to equally imply all knowledge media into the
 M. Kohlegger, R. Maier, S. Thalmann: Understanding ... 55
content analysis, identical numbers of models were drawn from each subset. The
sample based on this trisection finally contained 16 1 models [Table 1].

Approximated
Model Subset Acronym Source of information 2
Diversification
PSP 2120 [Humphrey, et al. 2005]
PCMM 561 [Curtis, et al. 1995]
Person SFIA 70 [SFIA 2007]
Dreyfus Model 31 [Dreyfus, et al. 1988]
Cross Model 0 [Cross 2007]
SPICE 971 [Coletta 1995]
UMM 138 [Earthy 1999]
Object EMM 77 [Marshall, et al. 2004]
SMMM 40 [April, et al. 2005]
bIMM* 19 [Chamoni, et al. 2004]
CMM 13600 [Paulk, et al. 1993]
CObIT 830 [ITGI 2007]
Nolan Model 602 [Nolan 1979]
Social System
TMM 184 [Burnstein, et al. 1998]
CM3 84 [Kajko-Mattsson 2002]
OIMM 37 [Clark, et al. 2001]
*
Model was added to the list subsequently

Table 1: The sixteen selected maturity models to be analysed

In order to maximise meaningfulness of the gained results, only the five most
cited models in each category were used as input for the content analysis. The ranking
was based on the number of search results returned by Google Scholar using the
model’s name as search term.

3 Study Results
On the basis of the structured content analysis applied to 16 selected maturity models
the following model characteristics could be found which are illustrated with the help
of a number of questions. Some characteristics show similarities, others show
differences between the analysed models.
A model characteristic was categorised as similar among the analysed models, if
the relative frequency of appearance of one code of the sub question was at least 65%.
Those sub questions that complied with this criterion are subsumed in [Table 2].

1
Initially, the sample contained five models from each category. During the analysis, one
model, suggested to describe the process of object maturing, was found wrongly classified. As
the model was already analysed, it had not been rejected from the analysis table. Instead, the
next model from the list of object maturing models was analysed.
2
Each model was analysed on the basis of two textual sources. Here, only the most recent of
these two sources is listed.
56 M. Kohlegger, R. Maier, S. Thalmann: Understanding ...

model name

Frequency (ABS)
Dreyfus model

Nolan Model
Cross Model

SMMM
PSMM
CObIT

OIMM

SPICE

UMM
CMM

EMM

TMM
SFIA
CM3

PSP
category and code
How do elements change in number 1 1 1 3
change in time? change in nature 1 1 1 1 1 1 1 1 8
increasing change 1 1 1 1 1 1 1 1 1 1 1 1 1 13
Direction of change
decreasing change 1 1
person competence 1 1 1 1 1 5
document 0
infrastructure 1 1
object
product 1 2
service 1 1
Maturing
group 0
object
team 0
social community 0
System process 1 1 1 1 1 5
routine 0
structure 1
no model is
1 1 1 1 1 1 1 1 1 1 10
Complemented Model complemented
CMM 1 1 1 1 4
iterative (one path) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14
Model design
cyclical (many turns) 0
upper level comprises
1 1 1 1 1 1 1 1 1 1 10
How do levels built on lower level
other levels upper level is new
1 1 1 3
concept
defined goals have to
How to come from one 1 1 1 1 1 1 1 1 1 1 1 11
be fulfilled
Level to another?
matures implicit 1 1 1 3
there is a "not
1 1 1 3
Is there a "not existent" existing" stage
stage? there is no "not
1 1 1 3
existing" stage
one trigger per stage 0
many triggers per
Degree of detail 1 1 1 1 1 1 1 1 1 1 10
stage
no triggers per stage 1 1 2
Number of Goal 1.1
metric values 1 1 1 1 2 1 1 1 1 2 1
Levels 7
Method of goal metric based 1 1 1 1 4
benchmarking non-metric based 1 1 1 1 1 1 1 1 1 1 1 1 12
conceptual model 1 1 1 5
Model use
applied model 1 1 1 1 1 1 1 1 1 1 1 10
experience from praxis 1 1 1 1 1 1 6
What is the model standards 1 1 2
description based on literature (e.g., other
1 1 1 1 1 1 1 1 1 1 10
models)
certification is
1 1 2
available
Certification available?
no certification
1 1 1 1 1 1 1 7
available

Table 2: Similarities among maturity models

If the relative frequency of appearance of one code within a sub question was
smaller than 65% the sub question was added to the differences list [Table 3]. The
purpose of this analysis is to inform the design of maturing models. In the following,
the main results are presented in the form of questions that can be posed when
designing maturity models which are answered on the basis of the example of the
knowledge maturity model presented by [Maier, et al. 2007]. The analysis increases
modelling precision and shows avenues for further development.
 M. Kohlegger, R. Maier, S. Thalmann: Understanding ... 57

Frequency (ABS)
Dreyfus model

Nolans Model
Cross Model
model name

SMMM
PSMM
CObIT

OIMM

SPICE

UMM
CMM

EMM

TMM
SFIA
CM3

PSP
category and code
change in quality 1 1 1 1 4
change in capability 1 1 1 1 1 1 1 7
What means maturing?
change in risk 1 1 2
other change 1 1 1 3
has no conceptual mother model 1 1 1 1 1 1 1 7
CMM 1 1 1 1 1 1 6
Conceptual Mother Model
SPICE 1 1 2
Other model 1 1 2
Organisation internal assessment
1 1 1 1 1 1 1 1 1 1 1 1 12
team
Model User Organisation external
1 1 1 1 1 5
assessment team
model is not used practically 1 1 1 3
no model is complemented 1 1 1 1 1 1 1 1 1 1 10
Complemented Model
CMM 1 1 1 1 4
5.5
Number of stages metric value 5 6 3 5 6 3 6 5 5 4 7 5 6 5 6
0
there is a "not existing" stage 1 1 1 3
Is there a "not existent" stage?
there is no "not existing" stage 1 1 1 3
trigger descriptions 1 1 1 1 1 1 1 1 1 9
activity descriptions (tasks,
What do level descriptions include? 1 1 1 1 1 1 1 1 1 9
processes)
conceptual level description 1 1 1 1 1 1 1 1 1 1 1 1 1 13
explicitly allowed 1 1 2
Level skipping
not recommended 1 1 1 3
Are there parallel maturing parallel maturing is possible 1 1 1 1 1 1 1 1 8
processes possible for one unit parallel maturing is not possible 1 1 1 1 1 5
interviews 1 1 1 3
Where do assessment data come documents 1 1
from questionnaire 1 1 2
data 1 1
supported by assessment model 1 1 1 1 1 1 6
Tool support supported by software tool 1 1
not supported by tool 1 1 1 1 4

Table 3: Differences among maturity models

What is the nature of maturing?

- How do elements change in time? Knowledge elements change in nature while
they advance through the knowledge maturing process.
- What does maturing mean? Maturing means change in formality, distribution,
commitment, legitimation, understandability and teachability of the maturing
subject, i.e. of socially constructed knowledge in an organisation.
- What is the direction of change? Maturing means an increase of formality,
distribution, commitment, legitimation, understandability and teachability.
- What is the maturing subject? The maturing subject is a knowledge domain in the
sense of knowledge about a topic in a socially distributed activity system.
How is the model designed?
- Has the model a conceptual mother model? The knowledge maturing model is not
related to any other maturity model.
- What is the model used for? The model can be used as an analytic model to help
structure the analysis of existing organisational and technical infrastructures in
support of goal-directed learning on a collective level.
58 M. Kohlegger, R. Maier, S. Thalmann: Understanding ...

- Who uses the model? The model can be used by people taking on the role of
guides helping to foster and reduce barriers for knowledge maturing.
- Does the model complement other models? The model does not complement
another model.
- How is the model designed? The model is designed as a sequence of phases which
do not necessarily build on each other. Each phase can use outputs provided by
all other phases as inputs for knowledge maturing activities in this phase.
- How do the stages build on each other? The stages of the model are independent
from each other as the upper stage cannot be traced back to the lower stage.
- How does the subject proceed from one level to the next? The maturing element
matures implicitly between the stages. However, there can be explicit decisions to
take a knowledge domain from one phase to the next one.
- What is the number of stages? The knowledge maturing model has five stages.
- Is there a „not-existing“-stage? The model has no “not existing”-stage.
- What do the level descriptions include? The level descriptions of the knowledge
maturing model include conceptual descriptions of the stages.
- What is the degree of detail of the trigger description? The model has no triggers
between its stages.
- Is level-skipping allowed? The model does not mention level skipping.
- Are there parallel maturing processes possible for one unit? Within one
organisational unit, there might be different knowledge domains which are in
different knowledge maturing stages.
- What is the number of goal levels? The model has no goals assigned to its stages.
- What is the method of goal benchmarking? The model uses a non-metric way of
goal benchmarking.
- Where do assessment data come from? The model is not practically used yet, so
assessment has not been specified yet.
How is the model used and supported?
- What is the model used as? The model is used as a conceptual model.
- Is tool support available? The model is not supported by a tool.
- What is the model description based on? The model is based on practical
experience.
- Is certification available? There is no certification available for the model.

4 Implications for maturity modelling

Aside from a first definition proposal given by [Ahlemann, et al. 2005, 15], the nature
of maturity models has not been theorised well in literature until now. Maturity
models are seen as models that reflect certain aspects of reality, often called
capabilities, and define qualitative attributes which are used to classify a competence
object into one of several clearly defined classes. These classes are typically brought
 M. Kohlegger, R. Maier, S. Thalmann: Understanding ... 59
into a sequential order [Ahlemann, et al. 2005, 13p]. Although several institutions
developing maturity models (e.g., SEI, PMI) have tried to close this gap, the
foundation of their contributions has often not been based on the richness of maturity
models available, but only focused on their own models.
This work intended to define maturity models on the basis of the empirical data
that was gained in the structured content analysis. Therefore, a sample of 16 maturity
models was investigated. The characteristics and patterns of the observed models
were extracted from the model description and put into a well arranged structure
through a content analysis. From this, the following definition of maturity model has
been developed:

A maturity model conceptually represents phases of increasing

quantitative or qualitative capability changes of a maturing element in
order to assess its advances with respect to defined focus areas.

Typically, the maturing element is a person, an object or a social system. The

focus area determines which indicators for maturity can be used to assess a maturing
element. Examples of focus areas are maturity of processes, of digital resources or of
people’s competencies. The model structures the maturing elements’ developments
into a suitable number of phases which are commonly separated by non-metric based
trigger conditions and brought into a sequential order. A maturing element can
develop into any phase of the model, if it fulfils all trigger conditions of the lower
phases plus the ones of the actual phase. It is also important to understand that
maturity models can be either used in a descriptive way explaining changes observed
in reality or in a normative way. In the latter sense, the maturity model is intended to
guide owners’, managers’ or other committed individuals’ interventions into making
changes in maturity of maturing elements more effective or efficient.

5 Conclusion
This paper targets a striking gap in theory building about maturity models and can be
understood as a first step into the investigation of the nature of maturity and maturity
modelling. The coding table, containing the questions to this analysis shows how
many aspects have to be accounted in connection with the creation of a maturity
model. These aspects can be used as checklist for selecting a maturing model, for
comparing maturity models or for designing maturity models.
Besides the already mentioned coding bias, especially a model selection bias
challenges the findings of this work, as the analysed sample of models determines the
codes that have been found during the analytical process. If the size or the
composition of the sample would be changed, the codes might also change. Thus, the
results of the structured content analysis, although already considered useful, e.g., for
designing maturity models for the domains of knowledge, knowledge-intensive
processes or knowledge infrastructures., can be used as input for a quantitative study
with a larger sample of maturity models. Propositions that can guide such a
quantitative research can build on the basis of the findings presented here.
60 M. Kohlegger, R. Maier, S. Thalmann: Understanding ...

Acknowledgement
This work was co-funded by the European Commission under the Information and
Communication Technologies (ICT) theme of the 7th Framework Programme (FP7)
within the Integrating Project MATURE (contract no. 216356).

References
[Ahlemann, Schroeder and Teuteberg 2005] Ahlemann, F., Schroeder, C., Teuteberg, F.:
"Kompetenz- und Reifegradmodelle für das Projektmanagement. Grundlagen, Vergleich und
Einsatz"; ISPRI - Forschungszentrum für Informationssysteme in Projekt- und
Innovationsnetzwerken Osnabrück (2005).

[April et.al. 2005] April, A., Desharnais, J.-M.: "Software Maintenance Maturity Model
(SMMM): A Software Maintenence Process Model"; Statistic Canada Conference, Ottawa
(2005),
[Burnstein et.al. 1998] Burnstein, I., Homyen, A., Grom R., Carlson, C.: "A Model to Assess
Testing Process Maturity"; Crosstalk. The journal of defense SOftware Engineering, 1998,
November (1998),
[Chamoni et.al. 2004] Chamon, P., Gluchowski, P.: "Integrationstrends bei Business-
Intelligence-Systemen, Empirische Untersuchung auf Basis des Business Intelligence Maturity
Model"; Wirtschaftsinformatik, 46, 2 (2004), 119-128.
[Clark and Moon 2001] Clark, T., Moon, T.: "Interoperability for Joint and Coalition
Operations"; Australian Defence Force Journal, 115, (2001), 23-36.
[Coletta 1995] Coletta, A.: "The Spice Project: An internal Standard for Software Process
Assessment, Improvement and Capability Determination"; In: Objective Software Quality, (ed),
Springer, Heidelberg (1995).
[Cross 2007] Cross, J.: "Informal Learning. Rediscovering the Natural Pathways That Inspire
Innovation and Performance"; Pfeiffer, San Francisco (2007).
[Curtis et.al. 1995] Curtis, B., Hefley W., Miller, S.: "Overview of the People Capability
Maturity Model"; Software Engineering Institute (1995).
[Dey 1993] Dey, I.: "Qualitative Data Analysis: A User-Friendly Guide for Social Scientists";
Routledge, London (1993).
[Dreyfus et.al. 1988] Dreyfus, H. L., Dreyfus, S. E.: "Mind over Machine. The Power of
Human Intuition and Expertise in the Era of the Computer"; Free Press, New York (1988).
[Earthy 1999] Earthy, J.: "Usability Maturity Model: Process"; Information Engineering
Usability Support Centres 2.2, (1999).
[Greenberg et al. 1974] Greenberg, E., Sørensen, A.: "Towards a Concept of psychosocial
maturity"; Journal of Youth and Adolescence, 3, 4 (1974), 329-358.
[Humphrey et.al. 2005] Humphrey, W., Pomeroy-Huff, M., Coannon, R., Seburn, M.: "The
Personal Software Process (PSP) Body of Knowledge"; Software Engineering Institute 1.0,
(2005).
[ITGI 2007] ITGI: "Cobit 4.1"; Information Technologie Gouvernance Institute Rolling
Meadows (2007).
 M. Kohlegger, R. Maier, S. Thalmann: Understanding ... 61
[Kajko-Mattsson 2002] Kajko-Mattsson, M.: "Corrective Maintenance Maturity Model:
Problem Management"; 18th IEEE International Conference on Software Maintenance, IEEE
Computer Society, Kyoto (2002), 486-491.
[Krcmar 2005] Krcmar, H.: "Informationsmanagement"; Springer, Berlin (2005).
[Lamnek 2005] Lamnek, S.: "Qualöitative Sozialforschung"; Beltz Psychologie Verlags Union,
Weinheim (2005).
[Maier 2007] Maier, R.: "Knowledge Management Systems"; Springer, Berlin Heidelberg
(2007).
[Maier et al. 2007] Maier, R., Schmidt, A.: "Characterizing Knowledge Maturing"; Professional
Knowledge Management, Gito, Potsdam, Germany (2007),
[Marshall et.al. 2004] Marshall S., Mitchell, G.: "Applying SPICE to e-Learning: An e-
Learning Maturity Model?"; Australasien Computing Education Conference, Dunedin (2004),
[Mayring 2008] Mayring, P.: "Qualitative Inhaltsanalyse. Grundlagen und Techniken"; Beltz,
Weinheim und Basel (2008).
[Nolan 1973] Nolan, R.: "Managing the Computer Resource: A Stage Hypothesis";
Communications of the ACM, 16, 7 (1973), 399-405.
[Nolan 1979] Nolan, R.: "Managing the Crisis in Data processing"; Harvard Business Review,
57, 2 (1979), 115-126.
[Paulk et al. 1993] Paulk, M., Curtis, B., Chrissis, M. B. Weber, C.: "Capability Maturity
Model for Software"; Software Engineering Institute 1.1, Pittsburgh (1993).
[Schambach-Hardtke 2005] Schambach-Hardtke, L: "Theoretische Hintergründe
Sozialwissenschaftlicher Forschungsmethoden"; In: Einführung in das Methodenspektrum
sozialwissenschaftlicher Forschung, S. Gahleitner, S. Gerull, B. Petuya-Ituarte, L. Schambach-
Hardtke and C. Streblow (ed), Schibri-Verlag, Milow (2005).
[SFIA 2007] SFIA: "Framework reference SFIA"; SFIA Foundation 3, (2007).