Journal of Medical Systems (2018) 42: 235

https://doi.org/10.1007/s10916-018-1097-0

SYSTEMS-LEVEL QUALITY IMPROVEMENT

Information System Maturity Models in Healthcare

Jorge Gomes 1 & Mário Romão 1

Received: 25 October 2017 / Accepted: 9 October 2018 / Published online: 16 October 2018
# Springer Science+Business Media, LLC, part of Springer Nature 2018

Abstract
The use of information systems in healthcare (HIS) has been recognised as having crucial importance in improving the efficiency,
cost-effectiveness, quality, and safety of medical care delivery. HIS has the potential to improve individuals’ health and providers’
performance by producing better quality, cost savings, and greater patient involvement in their own health. There have been two
major drivers for the HIS investments in healthcare: The ever-increasing burden from chronic disease with costs growing signif-
icantly faster and the recognition of the need for greatly improved quality and safety in health delivery. Maturity models (MM) are
based on the premises that people, organizations, functional areas and processes evolve through a process of development or growth
towards a more advanced maturity, going through a distinct number of levels. Through a state-of-the-art review of HIS, focused on
their maturity state, we identify and characterize a set of critical factors recognized as determinants in the context of HIS maturity.
The article identifies a broad spectrum of MM applied to the health sector and its characteristics and reinforces the belief that the
maturity of HIS can contribute to the quality of information and knowledge management in the sector.

Keywords Maturity models . Information Systems in Healthcare . Information Systems Investments in Healthcare . Information
systems health maturity models

Introduction between IS investments and business value which is still par-

tially inexplicable. Several empirical studies in the manage-
In almost every industry, organisations rely on investments in ment information systems field have shown that the use of
Information Systems (IS) to realise benefits after their success- computer-based applications could have positive impacts on
ful implementation [1]. Healthcare organisations face increas- organizational performance, including the healthcare organi-
ing pressure to improve their processes and to provide evi- sations [9, 10]. HIS have the potential to improve methods and
dence of the quality and efficiency of their operations [2]. processes that support the individual’s health, promote better
Despite the tremendous investments in IS, the studies present operational performance, and provide more and better quality,
contradictory evidence as to the expected benefits [3, 4]. The cost savings and patient involvement [11]. The health sector
relationship between investments in IS and organisational per- has found that the underlying reasons for a certain inadequacy
formance has been the subject of many discussions [5]. in the management of health processes is directly related to the
Although most of the research finds empirical evidence in limitations of infrastructure and its inefficient management
favour of the operational and strategic relevance of IS/IT [12, 13]. The more comprehensive the technology, or the
[6–8], the fundamental question lies in the causal relationship broader the extent of its implementation, the harder it seems
to achieve success [14, 15]. Governments have decided to
This article is part of the Topical Collection on Systems-Level reform their health care system to improve transparency, qual-
Quality Improvement ity, safety, patient satisfaction, and, above all, cost control.
Hospitals have invested heavily in improving and managing
* Jorge Gomes their processes. Although there is no consensus on the capac-
[email protected] ities that hospitals need to acquire to become process-oriented
organizations, nor a consensus on the sequence of steps I have
Mário Romão
[email protected] followed for their development [16]. Health sector, stimulated
by the successful implementations experienced in other indus-
1
ADVANCE, Rua Miguel Lupi, 20, 1200-781 Lisbon, Portugal tries, have developed some organisational models, such as, the
235 Page 2 of 14 J Med Syst (2018) 42: 235

patient-focused hospital or clinical pathways to introduce pro- variety of application systems in a hospital [10, 30, 31]. The
cess management into hospitals [17, 18]. Prior statistical re- completeness of the infrastructure is generally proportional to
search shows that process orientation significantly enhances the maturity of IS/IT of the hospital [22]. Many studies in IS
hospital performance [19]. Overall, this paper intends to con- have indicated that system integration is another important
tribute towards a deeper understanding of the process man- challenge for the development of IS within healthcare organi-
agement maturity in hospitals. sations [30, 31]. Several standards have been developed to
cope with the problem of integration for healthcare providers,
particularly regarding the exchange of data between health
Maturity models for information systems organisations [22, 32]. The overall idea is that organizations
in healthcare will increasingly adopt maturity models to stimulate and guide
the development of their IS capabilities.
Within the healthcare sector, several maturity models have
been developed, although these models are still at an early
stage of development [20]. The basic concept underlying ma- Methodology
turity is that mature organizations do things systematically,
while immature organizations achieve their outcomes because The methodology adopted for the literature review aims to
of the heroic efforts of individuals using approaches that they conduct a wide and comprehensive documental review. The
create and use spontaneously [21]. A hospital with a mature IS initial step is to provide the criteria to choose the approach
infrastructure can reflect higher degree of formalization of IS and establish the strategies to be applied to the current project.
planning and control processes [22]. Governments have Following the strategy proposed by [33], five steps were need-
attempted to reduce costs and increase productivity by ed to a systematic of the literature review. (1) Defines terms,
implementing enterprise resource planning systems or process keywords and combinations to be used as criteria to be applied
management systems [21]. To identify and explore the in the review (2). Identify relevant works that contain the key-
strengths and weaknesses of the organizational designs, a words and terms defined above (3) Promoting an assessment of
wide range of MM have been developed by practitioners identified papers and select the works that meet the criteria; (4).
and academics over the past years [23]. MM have become Extracting the relevant information from the selected literature.
an important topic in management research and are defined (5) Finally, perform a synthesis of data (Table 1).
as conceptual multi-stage frameworks that describe typical The research materials were collected initially across the plat-
patterns in the development of organizational capabilities forms AIS Electronic Library, ISI Web of Knowledge, SCOPUS,
[20]. MM are based on principle that people, processes, and Springer, Elsevier/Science Direct and IEEE Computer Society
organizations evolve to a higher level of capability following a Digital Library. Afterwards we moved to a more extended search
process of development, which encompasses an evolutionary through the search engine Google Scholar and Google to ensure
sequence of phases [24]. MM provide managers and organi- identification of other relevant work for the study. As a quality
sations with an important framework for identifying the capa- criterion it was established to gather all the studies where matu-
bility status of an IS, and to plan and implement actions that rity models were mentioned, directly or indirectly, and that clear-
allow them to advance to a higher maturity stage and thus ly identify the contextual factors, such as, motivation, goals,
achieve the proposed objectives. MM are a means to support results, and benefits (Table 2).
effective management and continuous improvement for initia- The characterization of each model was done taking into
tives that are complex and have multiple components [25, 26]. account the description, scope, stages and their characteristics,
While current research is occupied with the specification of size, influencing factors, methods adopted in the development
ever-more rigorous approaches for developing MM, there has and validation process. In the end, after processing of all cases,
been a general disregard of effectively implementing MM in 26 models were selected, which are described below.
complex organizational environments, such as hospitals [27].
Several models have been developed to help assess or de-
scribe the current level of IS adoption in the health sector Maturity models approaches
[28]. These are useful benchmarks when looking at patient
information systems. In the health sector, e-health services In the literature review described in the previous section, we
often involve multiple collaborating healthcare providers, in- identified 26 /Maturity Models. These models were summary
dividuals or organisations. Their ability to interoperate will described on the present section and listed in Table 4. Each
significantly influence their capability to deliver safe, reliable, model is characterized by its focus and domain, a sequential
efficient and convenient healthcare services [29]. The capabil- identification, its name and/or initials, a reference for the paper
ity of IS infrastructure is used as the basis for defining the where it is described, and a list of the source models on which
sophistication of IS in organizations, supporting a wide it is based.
J Med Syst (2018) 42: 235 Page 3 of 14 235

Table 1 Methodology

1. Defines terms and keywords

2. Identify works that contain the keywords and terms

3. Academic works that meet the criteria

4. Extract information from the selected documentation

5. Synthesis of data

Capability maturity model integration for services methodology and algorithms to automatically score hospitals
(CMMI-SVC) around the world relative to their EMR capabilities. This
eight-stage (0–7) model measures the adoption and utilization
One of the most prominent MM is the Capability Maturity of the EMR functions [35], moving the organizations closer to
Model Integration (CMMI) with its roots in software engi- achieving a near paperless environment that harnesses tech-
neering where it was found relatively helpful in guiding and nology to support optimized patient care by completing each
monitoring the maturity of software development practices stage below. An EMR is a multifaceted, electronic patient
[34] and over the years it was extended to other domains. records system and is often adopted in a model of hierarchical
CMMI is a process level improvement training and appraisal and increasingly complex stages [60]. Through EMRAM, pa-
program and introduces the concept of five maturity levels tient records become electronic and accessible across inpatient
defined by special requirements that are cumulative and uses and outpatient environments, and health care practitioners can
standardized question catalogues and evaluation criteria to document, monitor, and manage health care more effectively
assess an organizations product development process. [61]. The results of the EMRAM assessment can be used to
CMMI includes a self-assessment that presents the organisa- identify key opportunities for improvement, to drive an IT
tion’s best practices in key process areas and then shows how strategy and alignment with the overall business strategy of
the organisation can redefine its capabilities as it evolves into a an organization.
more mature state [34]. CMMI-SVC goals and practices are
therefore potentially relevant to any organization concerned
with the delivery of s health care services [34]. Electronic Healthcare Maturity Model (eHMM)

HIMSS analytics electronic medical record adoption In general, MM focuses mainly on the organisations, although
model (EMRAM) eHMM incorporates all the services providers associated to
the healthcare processes, adjustable to any specific provider
The EMRAM is an international standard to measure at any maturity level and able to show different levels of ma-
Electronic Medical Records (EMR) Adoption and is used by turity for different business processes [62]. The following pro-
over 9000 hospitals across the globe. It incorporates a cess maturity levels provide a roadmap for organizations
embarking on the journey of continuous process improve-
ment. The eHMM proposes a 7-level maturity model to map
Table 2 Search criteria how health processes can reach maturity. The model shows
Search criteria the evolution, improvement, and transformation of a business
Maturity Model AND Health
over time and captures its capabilities at each intermediate
Maturity Model AND Healthcare
level. The maturity model is used in contemporary methodol-
ogies for setting goals and measuring progress. The eHMM
Maturity Model AND Hospital
illustrates a transformation of the healthcare enterprise elec-
Maturity Model AND eHealth
tronic process from an immature to a national level. This is
Maturity Model AND HIS
explained through entities, departments and infrastructure at a
Maturity Model AND Health Information
System defined point in time. Each level has distinct characteristics
that differentiate it from other levels.
235 Page 4 of 14 J Med Syst (2018) 42: 235

IDC Maturityscapes technology, staffing, and other related investment decisions;

(4) Uncover maturity gaps among business units or between
IDC MaturityScapes were created by IDC Health Insights to business and IT groups; (Leverage mobile technology for sig-
provide a detailed explanation of the stages from the simplest, nificant long-term competitive advantage).
unstructured ad hoc stage to the advanced, systematized opti-
mized level, offering an opportunity for managers and their IDC healthcare IT maturity model – IT (HIT)
organisations to have a structured way to identify their current
level of capability, or maturity, and find the gap between IDC (Health Industry Insights) developed a maturity model
where they are and where they want to be to maintain com- that describes the five developmental stages of hospitals IS
petitive balance or achieve industry superiority. IDC (basic HIS, advanced HIS, clinical HIS, digital hospital and
MaturityScapes are designed to be used assuming change in virtual hospital) [38]. Each step is supported by the capabili-
the IT ecosystem — as a result of new technological advances, ties of the previous stage. Healthcare IT (HIT) Maturity
the maturation of existing technology use, or the integration of Model, has been used worldwide by IDC to assess the matu-
technologies, like social and mobile applications, that were rity of the hospitals IS (HIS) [38].
previously considered to be separate platforms. These maturi-
ty models are particularly intended for enterprises making Healthcare information technology maturity model
significant, business-transforming investments in the 3rd (HIT-MM)
Platform [36]. IDC MaturityScapes can be used to enable
managers to align business value goals with IT strategy, as a HIMSS Analytics USA/Europe and Innovation Value Institute
tool to identify where investments in people, process, and have come together to create an industry leading programme
technology can be consistent with what the organization re- for hospitals to enhance their IT organisational capabilities
quires. This is an input to a variety of business-IT dashboards towards achieving better eHealth outcomes [39].
that monitor and measure IT capabilities against best practices The HIT-MM is aimed at CIOs and senior IT decision-
and as a roadmap for overall improvement of IT processes, makers responsible for delivering and running clinical
communications, and business integration [36]. IDC eHealth systems as well as more traditional IT systems. The
MaturityScapes dimensions are five and were often associated results of the programme provide a solid foundation to trigger
with issues that arise from having to implement change to senior level decisions within the hospital in relation to improv-
people, technology, and processes (Intent: Strategy, sponsor- ing constrained IT organisational capabilities, which are con-
ship, and justification; Data: Relevance, quality, and availabil- sidered essential for delivering and running better HIS and
ity; Technology: Adoption, performance, and functionality; services [39]. The programme enables hospitals to map the
People: Skills, culture, and organizational structure; Process: maturity of their healthcare IT services and the maturity of
Tracking, analysis, and decisioning). Each of the five dimen- the organisational capabilities to deliver and run those services
sions points to several sub-dimensions (data describing rele- within IT. This unified approach reveals dependencies that
vance, quality, and availability). These are the key factors that may constrain the strategizing, deployment and running of
the model (in this case) uses to chart a course for data maturity clinical IT services, through tracking levels of EMR adoption
— which in turn supports a more coherent overall Big Data with underlying IT organisational capabilities (via the IT
strategy for the 3rd Platform. Capability Maturity Framework- IT-CMF identifies maturity
of IT organisational capabilities from ad-hoc to optimising)
IDC mobility maturity model (IDC-Mobility) [39].

Healthcare is undergoing a mobile transformation because of The telemedicine service maturity model (TMSMM)
the consumerization of technology and the digitization of pa-
tient health information [37]. Consumers want to use a mobile The conceptual telemedicine service maturity model
device to interact with their health plans and physicians and TMSMM was developed by following an iterative process
manage their health [37]. involving telemedicine practitioners from five different
The Maturity Model for Mobile in Healthcare prescribe South African provincial departments of health (DoH). The
five stages of mobile maturity (ad hoc, opportunistic, repeat- TMSMM is developed in response to the need for a frame-
able, managed, optimized). IDC-Mobility for healthcare pro- work according to which the maturity of existing and pro-
vides the building blocks for developing a roadmap for enter- posed telemedicine projects can be measured with the purpose
prise mobility. This framework is meant to enable healthcare of supporting decision making towards sustained telemedicine
organizations to [37]: (1) Assess mobility competency and services [40].
maturity; (2) Use the baseline to define short- and long-term The TMSMM is designed along three dimensions:
goals and plan for improvements; (3) Prioritize mobility eReadiness Categories; Telemedicine Process Steps; and
J Med Syst (2018) 42: 235 Page 5 of 14 235

Maturity Levels. The intercept of each pair of these dimen- community [29]. Provides a set of guidelines for setting
sions form a matrix, each with a specific significance and organisational process improvement goals in delivery of inter-
function. This dimension includes all micro-level, meso- operable e-health solutions and a point of reference for ap-
level and macro-level processes required to make this happen. praising an e-health organisation’s interoperability through
The maturity scale of the TMSMM is based on the generic the respective interoperability systems or work products [29].
level indicators of the capability maturity model CMM [40].
According the TMSMM, the success determinants are orga- Business process orientation maturity model
nized as follows [63]: (1) Technology and maintenance - ICT (BPOMM)
availability, reliability, training, usability; (2) Policy and leg-
islation - Governmental and institutional policies and proce- The BPOMM was developed to measure the process orienta-
dures, standardization and security; (3) Individual users - tion maturity of employees within a large hospital facility [41].
Trust and willingness of users and decision makers, producing The BPOMM measurement tool provides hospitals with a
evidence, change in way of doing; (4) Organizational process- means to evaluate their evolvement towards process orienta-
es - Decision making processes, work procedures; (5) tion maturity. The model describes the different stages through
Planning and financial sustainability – Business models which which an organization must go to reach the goal of being fully
will ensure continuation of the telemedicine endeavour; (6) process oriented (ad-hoc, defined, linked, integrated) [65].
Interaction and involvement with community. Telemedicine The model benchmarks each health unit with competitors or
is per definition the delivery of healthcare service over a dis- other organizations, based on their relative position in the
tance. This model can be used to measure, manage and opti- model. The measurement tool consists of 35 questions mea-
mize all the components of a telemedicine system, as well as suring seven dimensions, the seven dimensions can be further
the health system within which it is implemented [63]. subdivided into two parts [41]: the BPO-Components and the
BPO-Impacts. The BPO-components measure the process ori-
Continuity of care maturity model (CCMM) entation and included three dimensions (Process View – 4
items, Process Jobs - 3 items, Process Management and
CCMM is a strategic framework created by HIMSS to guide Measurement – 4 items). The BPO-Impacts include the re-
continuity of care implementation. This model was created to maining four dimensions. They determine whether the BPO
help optimise outcomes for health system and patient alike. score results in improved organizational performance and
CCMM outlines the progressive capabilities healthcare orga- long-term health.
nisations need to possess to seamlessly coordinate patient care Moreover, the MM can be further detailed by including the
across a continuum of care sites and providers. The CCMM is individual scores of each BPOMM component and their relat-
based on eight stages (0–7) and addresses the convergence of ed impacts. Some of the benefits reported in the literature are:
interoperability, information exchange, care coordination, pa- cost savings through a more efficient execution of work, re-
tient engagement and analytics with the goal of individual and duced cycle times, improved customer focus, better integra-
population health management [64]. Every organisation starts tion across the organization, increased flexibility of the firm
at Stage 0, moves into basic peer-to-peer data exchange along with improved customer satisfaction, elimination of re-
progressing to using discrete and structured data in Stages dundant and duplicated activities [66].
1–3. In Stages 4–6 the model looks for enhanced levels of
coordinated care and expanding circles of care provider and Process management maturity model (PMMM)
patient engagement. In Stage 7 organisations optimise all parts
of healthcare delivery with a focus on patient centred, dynam- Technology usage is insufficient to evaluate the IT level of an
ic, knowledge driven interconnected healthcare delivery [64]. organization. Other benchmarks, such as the attitude of staff
towards new technologies, should also be incorporated into
Interoperability maturity model (IMM) the assessment of organizational IT maturity, and Nolan’s
stage model is a well-known.
Individual health information must follow the patient as he IT model fulfilling these requirements. A lack of consensus
receives services from various providers. This requires data regarding the capabilities and development stages of hospitals
interoperability which is the key to effective use of health required to become process-oriented. Successful process man-
information. The National Australian E-Health Transition agement within hospital settings require a much stronger focus
Authority (NEHTA) has defined an Interoperability Maturity on both cultural and structural capability areas than it does in
Model (IMM) [29] that identifies increasing capability for data other organisations, where the focus is rather on IT-support
interoperability. IMM defines an iterative process by which e- and process automation [21]. The conceptual basis for the
health organisations can assess and increase their ability to PMMM consists of five capability dimensions, including:
interoperate, internally or as part of a national e-health (1) Culture - Covering communication and leadership-
235 Page 6 of 14 J Med Syst (2018) 42: 235

related practices; (2) Strategy - Covering principles that are The ICMM aim is to encourage leaders to develop organi-
prerequisite for a full development of process management; sations to: (1) appreciate the potential role of informatics in
(3) Structure - Comprising the organisational dimensions; (4) providing business advantage; (2) treat informatics as a stra-
Practices - Summarising work practices that are crucial for tegic resource; (3) make investments in informatics aligned
process management; (5) IT - Including items that capture in with the business strategy; (4) deliver informatics enabled
how far the employed hospital IT systems can support a change effectively to optimise the business value.
smooth flow of complete patient care.
PACS maturity model (PMM)
NHS infrastructure maturity model (NIMM)
PACS has become an integrated component of today’s
healthcare delivery system and therefore can be considered
The NIMM framework supports the National Health Service
as the fundamental infrastructure for digital diagnostic imag-
(NHS) IT organisations to carry out an objective assessment of
ing and information management systems [44]. The PMM
their IT infrastructure and to identify infrastructure maturity
describes the process maturity of hospitals based on PACS.
improvement projects. The NIMM framework is divided into
PMM is a descriptive and normative model developed as a
13 categories, 74 capabilities, five perspectives and several
guide for evaluation and strategic planning through five levels
KPIs, each of which is scored out of five [42]. Not all capa-
of maturity [67]. This model can help hospitals to gain insights
bilities have to be completed at once. Review the capability
into their strategic objectives for growth and maturity about
list, decide the priorities for your IT organisation today and
PACS, the electronic patient record and other health informa-
concentrate your efforts on completing this subset. Each cat-
tion systems [67]. PMM combines three concepts [45]: (1)
egory is further divided into many capabilities which are used
PACS maturity as the concept to define PACS and its ele-
to target the assessment to a specific area. A capability is then
ments; (2) PACS alignment as the concept to complement
further organised into perspectives. Each perspective has sev-
the organizational dimensions of PACS; (3) PACS perfor-
eral KPIs associated with it, against which the capability in
mance as the added value of PACS within hospitals.
question is assessed. Organising the metrics into perspectives
Moreover, the proposed model can be applied as a valuable
provides the opportunity to review the capabilities and devel-
trool for organizational assessments, monitoring and
op an overall view of the capability rather than just from a
benchmarking purposes.
technology view point.
Health information network maturity model (HIN)
Informatics capability maturity model (ICMM)
The HIN is one of several tools that will enable HIN planners
The ICMM helps leaders to assess the role informatics play in and operators to operationalize the leading practice findings
their organisation to deliver business value. The ICMM as- and recommendations. It will allow HIN planners and opera-
sesses how well an organisation collects and manages and tors to objectively assess themselves, and to develop plans for
shares information, manages information and communica- enhancing their operational capabilities, and the level of ser-
tions technology, implementation and change; manages data vice and value they deliver [46]. The HIN has been
quality and governance; and uses health business intelligence synthetized into 10 capability domains (vision & engagement,
to achieve multidisciplinary integrated care [27]. The ICM governance, policy & legislation, skills & resources, financ-
instrument categorises the following key informatics capabil- ing, model practice, success metrics, clinical use cases, tech-
ity [43]: (1) Basic – Systems and processes not completely nology & apps, security & privacy). Five maturity levels have
reliable or coordinated; (2) Controlled – Systems coordinated, been identified for each capability domain (initial, anticipate,
manageable, performs consistently, but knowledge silos still interoperate, collaborate, optimize). The HIN comprises a set
exist; (3) Standardised – Standards used to support sharing of structured levels that describe the behaviours, practices,
and collaboration; (4) Optimised – Consolidated, efficient, processes, capabilities and milestone achievements which a
accountable with good governance; (5) Innovative - HIN will develop over time to reliably and sustainably deliver
Facilitates innovation with enterprise level engagement. The the infrastructure and services required to support health and
dimensions considered by the ICM are [43]: ICM1. Data col- health care related outcomes for a province or territory [46].
lection, integration and management in HIS/HER; ICM2. At a high level, the model comprises four components [46]:
Information sharing in the health neighbourhood; ICM3. (1) Maturity Levels - A five-level maturity continuum where
Managing health information and communication technology the uppermost level is a notional ideal state; (2) Domains - A
implementation and change; ICM4. Data quality management domain or key process or capability area identifies a cluster of
and information governance; ICM5. Using health business related activities or business capabilities that, when performed
intelligence to improve care and population health. together or combined, achieve a set of goals considered
J Med Syst (2018) 42: 235 Page 7 of 14 235

important to the success of one or more stakeholder groups; and IT infrastructure) identified as the most relevant for
(3) Goals - The goals of a key process or capability area healthcare IS [62]. These factors emerge as reference descrip-
summarize the states that must exist for that key area to have tors or variables that characterize each stage and determine the
been implemented in an effective and lasting way; (4) necessary criteria to reach a specific maturity stage. The
Characteristics - Common characteristics that implement and HISMM architecture comprehends levels on an evolutionary
institutionalize a key process area or capability. scale with measurable transitions between levels. Each level is
defined by a set of attributes, and when an HIS reveals such
The healthcare analytics adoption model (HAAM) attributes, the corresponding level and the capabilities it em-
bodies have been achieved. With measurable transition states
The Health Catalyst announced in 2012 a framework to mea- between levels. The stages are: Adhocracy, Starting the foun-
sure the adoption and meaningful use of data warehouses and dations, Centralized dictatorship, Democratic cooperation
analytics in healthcare in ways similar to the well-known Entrepreneurial opportunity and Integrated relationships.
HIMSS Analytics EMRAM model. The data-collection phase Hospitals can use this scale to: (1) define the current maturity
characterized by the urgent deployment of EMRs, will not stage; (2) determine the next achievable maturity stage; (3)
have a significant impact on the quality or cost of healthcare identify the attributes that must be met to achieve a new ma-
[47]. The HAAM was designed to ensure that organizations turity stage [68].
establish a foundational understanding of analytic technology
and organizational use of analytics in step-wise fashion before Healthcare usability maturity model (UMM)
attempting the more complicated topics of the upper levels.
Each level of adoption includes progressive expansion of an- Three accepted goals of usability are improved effectiveness,
alytic capability in four critical dimensions [47]: (1) New Data efficiency and user satisfaction. EHRs with poor usability can
Sources - Data content expands as new sources of data are impact clinician productivity [51]. Organizations typically in-
added to the healthcare ecosystem; (2) Complexity - stall EHRs to reduce errors. Incorporating usability into orga-
Analytic algorithms and data binding become progressively nizational processes could address these major detrimental
more complex (3) Data Literacy - Organizational data literacy impacts on clinicians. The authors co-led Usability
increases among employees, leading to an increasing ability to Taskforce with HIMSS develop this Maturity Model [69].
exploit data as an asset to organizational success, including The development of this model was based on the evaluation
new business and economic models; (4) Data Timeliness - of the characteristics of three usability maturity models and
Timeliness of data content increases which leads to a reduc- how they could be adopted in healthcare [48–50]. The UMM
tion in decision cycles and mean time to improvement. The identifies key elements and milestones involved in successful-
HAAM prescribed 8 maturity levels [47]: Level 8 - ly integrating usability into a healthcare organization. Each
Personalized Medicine & Prescriptive Analytics; Level 7 - phase allows organizations to identify their current level of
Clinical Risk Intervention & Predictive Analytics; Level 6 - usability and also includes guidance to move to the next stage.
Population Health Management & Suggestive Analytics; The five phases are the following [51]: (1) Unrecognized -
Level 5 - Waste & Care Variability Reduction; Level 4 - Lack of awareness of usability; (2) Preliminary Sporadic in-
Automated External Reporting; Level 3 - Automated clusion of usability; (3) Implemented - Recognized value of
Internal Reporting; Level 2 - Standardized Vocabulary & usability; (4) Integrated - All benchmarks of usability imple-
Patient Registries; Level 1 - Enterprise Data Warehouse; mented; (5) Strategic - Business benefit well understood, us-
Level 0 - Fragmented Point Solutions. Current adoption rates ability mandated, budget and people part of each year’s bud-
of data warehousing and analytics stand at only 10% and just a get, results used strategically throughout the organization.
small subset of those early adopters operates above Level 3
and none operates consistently above Level 5. Organisations Hospital cooperation maturity model (HCMM)
consistently reports that operates between Levels 2 and 3, no
higher [47]. HCMM took the following design decisions to build the ma-
turity model [52]:
Hospital information system maturity model (HISMM)
Opportunity - HCMM addresses an emerging phenome-
The HISMM was developed to address the complexity of HIS non, i.e. intensified networking and cooperation in
and offer a useful tool for the demanding role of HIS manage- healthcare. HCMM is a completely new maturity model.
ment. The HISMM displays a conventional Maturity Model But the structure is similar to a CMM-like model.
structure, that is, a matrix comprised by different maturity Scope - HCMM covers a very specific area of interest.
stages and six influence factors (data analysis, strategy, peo- Focuses on intra-organizational as well as inter-
ple, electronic medical record, information security, systems organizational aspects relevant to optimizing cooperative
235 Page 8 of 14 J Med Syst (2018) 42: 235

structures and processes in hospitals. HCMM is intended processes that include people, technology, and organizational
to primarily support decision making of hospital processes; (5) Incorporating aspects of quality including sys-
managers. tem quality, information quality, and service quality; (6)
Design Model/Maturity Concept - HCMM uses a multi- Providing an understanding of relationship between the differ-
dimensional approach to measure maturity, including ent levels and key processes involved in a maturity model by
items for strategic, organizational, and technical capabil- incorporating theoretical underpinnings.
ities as well as Bas-is^ and the targeted Bto-be^ maturity. The approach used for creating the requirements to include
Design activity/ Decision parameter - HCMM tries to in a maturity model will be validated empirically to confirm
identify challenges for cooperation and supports optimi- accuracy and completeness in the healthcare environment.
zation of cooperation in a holistic manner, thus underlies Gastaldi et al. [54] mapped BI considering dimensions and
multi-dimensional goals. HCMM measurement items are metrics in 4 maturity levels through the several functional
a combination of theory-driven and practitioner-based areas.
inputs. The creation of a maturity model for BI in healthcare has
Evaluate design/ Subject of evaluation - HCMM was great opportunity for contribution to information and knowl-
evaluated in terms of form and content (design product), edge management in healthcare. The overarching need for a
ex-ante (questionnaire) and in a naturalistic setting by maturity model for BI is to provide guidance to BI deployment
means of the experiences of real users. initiatives and serve as a readiness assessment to move up
each level in maturity. A maturity model can provide a read-
The HCMM is structured in three layers or dimensions iness assessment and planning for a BI strategy by providing
[52]: (1) A strategic layer -Used to measure the ability of a the insight to the critical steps and processes needed to reach a
hospital to cooperate with external partners; (2) desired level in BI maturity.
Organizational layer – Used to measure the ability to cooper-
ate within the hospital; (3) Information layer - Used to mea- BMeaningful use^ (Forrester model)
sure the technical capabilities of a hospital to provide the IT
infrastructure needed for internal and external cooperation ef- A wide variety of systems, organizations, and processes are in
ficiently and effectively. place to manage medical records and surrounding work pro-
cesses. Providers will take different paths at different speeds to
Business intelligence maturity model (BIMM) meet these challenges. This diversity led to a three-phase ma-
turity model that leverages content, collaboration, and
Today’s healthcare decisions makers are facing growing de- workflow technologies as building blocks for transition. The
mands for both clinical and administrative information [70]. Forrester model helps providers assess their content, collabo-
When evaluating Business Intelligence (BI) in the context of ration, and workflow state, and more importantly, determine
healthcare, it is important to understand the complexities and the road map required to get to the next phase. The three
how BI needs may be impacted. Three key areas that make phases are [55]: Phase 1: Paper- or imaged-based patient re-
healthcare BI efforts particularly challenging, namely: (1) The cords dominate. Most providers are in this mode, looking at
need for integration of clinical and financial data; (2) The patient information and a records or content management
diverse types of data formats that may provide information problem; Phase 2: Access to standalone repositories improves.
for higher level analytics; (3) The demands and expectations Providers have more patient information contained in the elec-
of external data for clinical and financial decisions. tronic medical records system with less dependence on paper;
The requirements for a BI maturity model for healthcare Phase 3: Access to the complete digital medical record is role-
were developed after a thorough literature review of existing based. Providers in Phase 3 exchange data electronically with
BI maturity models, processes and complexities in healthcare other providers, patients, and administrative systems. Content
information management, and critical success factors for BI is organized to support Bresults-based^ analysis.
success. The intended user of the BI maturity model would be According to Clair [55], this three stages model includes
management staff within a healthcare organization. Therefore, four dimensions or influencing factors: access, interoperabili-
it is important that the requirements are very practical so that ty, content features and planning and strategy.
the organization understands its maturity level after the eval-
uation, namely [53]: (1) Providing a conceptual structure for Health game maturity model (HGMM)
managing the use of business intelligence in healthcare; (2)
Focusing on the needs of operational/financial and clinical Gaming can be seen as the next step in the application of
information; (3) Focusing on capturing key business intelli- information technology to healthcare. Gaming improves inter-
gence processes and practices, taking into consideration spe- action and has a direct impact on human behaviour. The
cific processes within healthcare; (4) Incorporating key HGMM is developed to help healthcare organizations
J Med Syst (2018) 42: 235 Page 9 of 14 235

Table 3 – Dimensions across the functional areas

Functional Technological Diffusional Organisational

F1- Goal definition T1- BI architecture D1- Accessing users O1- BI strategy
F2- Measurement T2- Reporting D2- System users O2- BI budget
F3- Gap analysis T3- Interface D3- Process coverage O3- Organisation coverage
F4 -Data quality T4- User profiling O4- Key-user capabilities
F5- Functional integration T5- Technological integration O5- User capabilities
T6- Standards O6- Competence improvement
T7- Data provisioning O7- Partner/supplier coordination

introduce and improve their use of gaming. HGMM can help (3) Entity – Gamification is applied within functional groups
identify the position of a healthcare institution concerning or healthcare departments; (4) Institution – Gamification is
gamification and help the organization identify steps to pro- applied across the healthcare level; (5) Ecosystem –
ceed to the next maturity level. For instance, gamification can Gamification is applied in complete horizontal and vertical
be applied to make a task more fun for employees, without integration throughout the ecosystem of the users and pro-
necessarily achieving benefits like behavioural change or viders of healthcare.
learning. This would indicate a lower level of maturity in The fourth perspective of the model is related to the differ-
comparison to an organization that implemented gamification ent types of games that are used by organizations - Type: (1)
in its organization while achieving these benefits. In more Off-line – Non-internet, non-network components, such as
mature organisations, gamification contributes to actual traditional board games, are used; (2) Single-player – Online
healthcare execution. HGMM distinguishes four perspectives, single-player techniques are used; (3) Multi-player – Online
each with five maturity levels (based on the CMM approach) multi-player techniques are used; (4) Group playing – Online
[56]. group playing techniques are used; (5) MMO – Mass multi-
HGMM propose that organizations rank themselves on player techniques are used.
each of the four perspectives in order to assess the level of
gamification and the contribution of gamification to their stra-
tegic goals. Maturity model of hospital (MMH)
The first perspective of the model is related to the value
generated through gamification - Value: (1) Non-existent – No MMH summarizes the evaluation criteria performed by a hos-
value seen in using games, or no gamification in place; (2) pital to evaluate the level of maturity using a Balanced
Pleasure – Using games for pleasure but not yet for results. Scorecard (BSC), which is used when stand a strategy, and
Games have no real purpose; (3) Passion – Games promote the framework of CMMI, which is one of process maturity
flow and engagement and stimulate learning and healthcare model. MMH can be considered as an indicator of the strategy
simulation; (4) Purpose – The goal is to advance and innovate for improving hospital and learning the degree of improve-
using games. Actual healthcare execution and treatments are ment that can be achieved at each stage [57]. The framework
done through gamified processes; (5) Healthcare profit – has the following structure: (1) A BSC is a framework that has
Quality of healthcare is increased, and costs are lowered due been used well as a strategy in managed companies in recent
to use of gamification. years. It is used also for planning stage of a PDCA cycle. BSC
The second perspective of the HGMM is related to the consists of four perspectives (financial, customer, internal pro-
process that is in place to support gamification - Process: (1) cesses and learning and growth). Generally, learning &
Ad hoc - Gamification processes are unpredictable, poorly growth perspective is the foundation of a project, and it grows
controlled and reactive; (2) Repeatable - Processes are up with the internal process and the customer perspectives.
matched to projects and are often reactive; (3) Defined - Ideally it will be achieved in financial perspective.
Gamification processes are developed organization-wide and The MM is a measure that shows how well the organization
are proactive (4) Managed - The gamification process is mea- is managed (systematization). CMMI (Capability Maturity
sured and controlled; (5) Optimized - There is a focus on Model Integration) is one MM. Hospital Functional
continuous monitoring and improvement. Evaluation criteria (HFE) is divided into 6 common domains
The third perspective of the model is related to coverage, or and 2 peculiar domains. Moreover, the evaluation criteria are
the ways in which gamification and games are applied by divided into large, middle, and small items in each domain.
organizations - Coverage: (1) None – No gamification in Some total numbers of common items of the three above are
place; (2) Individual – Gamification is applied by individuals; 72 items, 178 items, and 577 items respectively.
Table 4 Summary of maturity models for healthcare
235 Page 10 of 14

Model name Lv Health focus Dimensions/ factors Ref. model Author

1 Capability Maturity Model Integration for Services (CMMI-SVC) 5 health care services 24 process areas CMM/ CMMI [34]
2 HIMSS Analytics Electronic Medical Record Adoption Model 8 EMR HIS application [35]
(EMRAM)
3 Electronic Healthcare Maturity Model (eHMM) 7 General Timeliness of process; Data access and accuracy of data; Process Effort; [13]
Cost effectiveness; Quality of process results; Utility or value to stakeholders:
4 IDC Maturity Scapes 5 3rd Platform technologies Intent; Data; Technology; People; Processes CMM/ CMMI [36]
5 IDC Mobility Maturity Model (IDC-Mobility) 5 Mobility Intent; technology; people; processes [37]
6 IDC healthcare IT maturity model – IT (HIT) 5 General HIS application [38]
7 Healthcare Information Technology Maturity Model (HIT-MM) General HIS application [39]
8 The Telemedicine Service Maturity Model (TMSMM) 5 Telemedicine Man; Machine; Material; Method; Money CMM/ CMMI [40]
9 Continuity of Care Maturity Model (CCMM) 8 EMR HIS application EMRAM [35]
10 Interoperability Maturity Model (IMM) 5 Interoperability Organisation, Information, Technical CMM/ CMMI [29]
11 Business Process Orientation Maturity Model (BPOMM) 4 Process Orientation Seven dimensions subdivided into two parts: the BPO-Components and the BPO-Impacts [41]
12 Process management maturity model (PMMM) 5 Process management Culture: Strategy; Structures; Practices; IT Nolan’s stage [21]
model
13 NHS Infrastructure Maturity Model (NIMM) 5 General People & Organisation; Technology; Security & Information Governance; Alignment & CMM/ CMMI [42]
Value
14 Informatics Capability Maturity Model (ICMM) 5 Infrastructure IT Managing IS; Using BI; Using IT; (4) Aligning Business and Informatics; Managing NIMM [43]
Change.
15 PACS Maturity Model (PMM) 5 PACS Strategy & Policy; Organisation & Processes; Monitoring & Control; IT; People & CMM/ CMMI [44]
Culture;
16 Health Information Network Maturity Model (HIN) 5 health information Vision & Engagement; Governance, Policy & Legislation; Skills & Resources; Financing; [45]
exchange Model practice; Success metrics; Clinical use cases; Technology & Apps; Security
& privacy.
17 The Healthcare Analytics Adoption Model (HAAM) 9 Data Warehouse & Analysis New Data Sources; Complexity; Data Literacy; Data Timeliness EMRAM [46]
18 Hospital Information System Maturity Model (HISMM) 6 General Data Analysis; Strategy; People, Electronic medical Record; Information security; [47]
Systems
and IT infrastructure
19 Healthcare usability maturity model (UMM) 5 Usability Focus on users; Management; Process & Infrastructure; Resources; Education [48–50] [51]
20 Hospital Cooperation Maturity Model (HCMM) 4 Networking /Cooperation Strategic; Organisational, Information CMM/ CMMI [52]
21 Business Intelligence Maturity Model (BIMM) 4 Business Intelligence Collaboration; Knowledge; Trust; Institutions; Governance [53, 54]
22 BMeaningful Use^ (Forrester Model) 3 EMR Access; Interoperability; Content Features; Planning & Strategy [55]
23 Health Game Maturity Model (HGMM) 3 Gamification Value; Process; Coverage; Types CMM/ CMMI [56]
24 Maturity Model for Hospital (MMH) 5 Medical service Learning & Growth; Hospital’s Process; Patients; Citizen CMM/ CMMI [57]
improvement
25 High Reliability Health Care Maturity Model (HRHCM) High reliability health care Leadership, Safety Culture; Robust Process Improvement. [58]
26 Healthcare Data Quality Maturity Model (HDQM2) Data quality Accuracy/ Correctness; Completeness; Uniqueness; Duplicates CMM/ CMMI [59]
J Med Syst (2018) 42: 235
J Med Syst (2018) 42: 235 Page 11 of 14 235

The strategy of MMH is the maintenance of a local medical (Accuracy/ Correctness, Completeness, Uniqueness and
system. This strategy considers what Ba local resident is effi- Duplicates). The Metamodel was represented by the following
ciently provided with when receiving high quality medical elements: Levels of Maturity, Practices, Process Areas and
service.^ Value Creation. For areas of process of the Maturity Model
MMH creates the domain rule with the four perspectives of the phases of data life cycle were considered.
BSC: (1) Learning & growth; (2) Hospital’s Process (3)
Patients; (4) Citizen.
The Maturity Levels of MMH are made into six stages, Discussion
from L0 (stage which is not carried out) to L5 (optimized
stage) based on CMMI. The Maturity Level rules of MMH The literature review identified a set of MM with different
follows 3 basic rules: (1) Bmw^ (Maturity model keyword) is focus and characteristics applied to healthcare. Some are more
the rule that transposes each requirement for a level of CMMI generalist and have a broader focus on HIS, namely, eHMM,
to the requirements for a medical system; (2) Bmp^ (Medical CCMM or IT (HIT), others, such as the EMRAM and
Priority) is the rule that carries out the level attached to the Meaningful Use models are examples of EMR applications.
priority of a medical act; (3) Bhfa^ (Hospital function average) Besides, there are MM that assess different aspects of HIS,
is based on the average of functional evaluation. namely: PACS (e.g. PMM); usability (e.g. UMM); interoper-
ability (IMM); platform mobility and devices (e.g. IDC
High reliability health care maturity model (HRHCM) Mobility); cooperation & networking (HCMM); business in-
telligence (e.g. BIMM); warehousing and data analysis (e.g.
Despite serious and widespread efforts to improve the quality HAAS); telemedicine (e.g. TMSMM); gaming (HGMM) and
of health care, many patients still suffer preventable harm medical service (MMH). We found also some national health
every day [58]. The lack of a tool for categorizing and differ- services that developed their own maturity models, namely the
entiating hospitals according to their high reliability organiza- UK National Service with the NHS Infrastructure Mature
tion (HRO)–related characteristics has hindered progress to- Model (NIMM) focus on the assessment on their IT infrastruc-
ward implementing and sustaining evidence-based HRO prac- ture and the National E-health Transition Authority of
tices. Hospitals would benefit both from an understanding of Australia with the NeTHA model focus on the
the organizational characteristics that support HRO practices interoperability.
and from knowledge about the steps necessary to achieve Most of the MM has their roots in the CMM/CMMI and the
HRO status to reduce the risk of harm and improve outcomes documentation describes their characteristics and specificity.
[71]. The HRHCM is a model for health care organizations’ Regarding the number of maturity stages, there are models
achievement of high reliability with zero patient harm, incor- from 3 stages as the BMeaningful Use^ model [55] up to 9
porates three major domains critical for promoting HROs— stages of HAAM [47] and not all the identified MM with
Leadership, Safety C ul ture, and Robust Process various dimensions or influencing factors have explicitly bro-
Improvement. ken down the characteristics for each stage of maturity. The
study also shows a wide range of generic and domain-specific
Healthcare data quality maturity model (HDQM2) MM that have been developed to improve organizational de-
sign and learning of healthcare organizations. Following we
Data Quality (DQ) is a central issue within the development of summarize the main features of the MM approaches (Tables 3
Healthcare Systems, for both the delivery of the service as and 4).
well as for the establishment of public policies [59]. There is
an overwhelming amount of existing evidence that DQ in
registered patients contains deviations and errors that should Conclusions
be evaluated and improved. If improvement does not happen,
these problems could generate adversity in the provided health Many hospitals and health systems are adopting delivery sys-
service. HDQM2 was developed taking as basis models found tem reform with the goal of better aligning provider incentives
within literature, considering the peculiarities of the Health to improving the patient experience of care, improving the
Sector and the characteristics of the local context. HDQM2 health of populations, and reducing the per capita cost of
starts from the scenario that an information system is com- health care. Every day it becomes clearer the importance of
posed of basic and fundamental elements for its characteriza- HIS in-health organizations, which deal with a multitude of
tion. These are: People, Processes, Data and Technology. This administrative and medical data of many patients. More than
model is formed by the following elements: Levels of having access to these data, the important thing is to transform
Maturity, Practices, Process Areas and Value Creation. The them into intelligent information that allows for further anal-
principal dimensions were extracted from DQ literature ysis that makes management more efficient and contributes to
235 Page 12 of 14 J Med Syst (2018) 42: 235

the improvement of the quality of care and provides financial 8. Ramirez, R., Melville, N., and Lawler, E., Information technology
infrastructure, organizational process redesign, and business value:
benefits to the organization. Having a clear strategy, an ade-
An empirical analysis. Decis. Support. Syst. 49(4):417–429, 2010.
quate governance system, optimized processes and a qualified https://doi.org/10.1016/j.dss.2010.05.003.
team makes it possible to take advantage of the full potential 9. Devaraj, S., Ow, T. T., and Kohli, L., Examining the impact of infor-
offered by the available technology. MM are no longer direct- mation technology and patient flow on healthcare performance: A the-
ory of swift and even flow (TSEF) perspective. J. Oper. Manag. 31:
ed exclusively at the evaluation of software vendors or soft-
181–192, 2013. https://doi.org/10.1016/j.jom.2013.03.001.
ware development processes. They are now commonly used 10. Paré, G., and Sicotte, C., Information technology sophistication in
as a means of benchmarking, self-assessment, change man- health care: An instrument validation study among Canadian hos-
agement, and organisational learning. Increasing maturity of pitals. Int. J. Med. Inform. 63:205–223, 2001. https://doi.org/10.
1016/S1386-5056(01)00178-2.
these processes and protocols in hospital management im-
11. Buntin, M. B., Burke, M. F., Hoaglin, M. C., and Blumenthal, D.,
proves the organisation’s profitability by making them more The benefits of health information technology: A review of the
efficient. Additionally, very demanding and exogenous regu- recent literature shows predominantly positive results. Health Aff.
lations can also benefit from the existence of MM at the level 30(3):464–471, 2011. https://doi.org/10.1377/hlthaff.2011.0178.
12. Berg, M., Implementing information systems in health care organi-
of organizational processes and the supporting HIS. CMM
zations: Myths and challenges. Int. J. Med. Inform. 64:143–156,
served as inspiration for dozens of MM. Some MM are devel- 2001. https://doi.org/10.1016/S1386-5056(01)00200-3.
oped by health national and supranational organizations, 13. Sharma, B., Electronic Healthcare Maturity Model (eHMM). White
mainly corporations, who are dedicated to technological de- Paper. Quintegra Solutions Limited, 2008. http://www.
velopments, such as IDC Health Insights and HIMSS or even quintegrasolutions.com/eHMM%20White%20Paper.pdf.
Accessed Sep 2018.
by national health organizations as the NHS or NEHTA. 14. Berg, M., Rationalizing medical work. Decision support techniques
and medical practices. Cambridge: MIT Press, 1997.
Compliance with ethical standards 15. van Gemert-Pijnen, J., Nijland, N., van Limburg, M. et al., Holistic
framework to improve the uptake and impact of eHealth technolo-
gies. J. Med. Internet Res. 13(4):e111, 2011. https://doi.org/10.
Conflict of interest The authors declare that they have no conflict of
2196/jmir.1672.
interest.
16. Cleven, A., Winter, R., Wortmann, F., and Mettler, T., Process manage-
ment in hospitals: An empirically grounded maturity model. Bus. Res.
Human and animal rights and informed consent This article does not 7(2):191–216, 2014. https://doi.org/10.1007/s4068.
contain any studies with human participants or animals performed by any 17. Bragato, L., and Jacobs, K., Care pathways: The road to better
of the authors. For this type of study formal consent is not required. health services? J. Health Organ. Manag. 17(3):164–180, 2003.
https://doi.org/10.1108/14777260310480721.
18. Rohner, P., Achieving impact with clinical process management in
hospitals: An inspiring case. Bus. Process. Manag. J. 18(4):600–
References 624, 2012. https://doi.org/10.1108/14637151211253756.
19. Cleven, A., Winter, R., and Rohner, P., The performance impact of
1. Gomes, J., and Romão, M., Maturity, benefits and project manage- process orientation in hospitals: Theoretical model and empirical
test. Research Report, Institute of Information Management,
ment shaping project success. In: Vol. 353 of the series Advances in
University of St. Gallen, 2011.
Intelligent Systems and Computing, pp 435–448, New
Contributions in Information Systems and Technologies, Vol. 1, 20. Harmon, P., Evaluating an Organization's Business Process
2015. https://doi.org/10.1007/978-3-319-16486-1_43 Maturity. Business Process Trends, 2, 3, 2004. https://www.
bptrends.com/publicationfiles/03-04%20NL%20Eval%20BP%
2. Kujala, J., Lillrank, P., Kronström, V., and Peltokorpi, A., Time-based
20Maturity%20-%20Harmon.pdf. Accessed Sep 2018.
management of patient process. J. Health Organ. Manag. 20(6):512–
21. Liu, C.-F., Hwang, H.-G., and Chang, H.-C., E-healthcare maturity
524, 2006. https://doi.org/10.1108/14777260610702262.
in Taiwan. Telemed. J. E. Health. 17(7):569–573, 2011. https://doi.
3. Tam, K. Y., The impact of information technology investments on org/10.1089/tmj.2010.0228.
firm performance and evaluation: Evidence form newly industrial- 22. Waring, T. S., Information management and technology strategy
ized economies. Inf. Syst. Res. 9(1):85–98, 1998. development in the UK's acute hospital sector: A maturity model
4. Devaraj, S., and Kohli, R., Information technology payoff in the perspective. Public Money Manage. 35(4):281–288, 2015. https://
health-care industry: A longitudinal study. J. Manag. Inf. Syst. 6(4): doi.org/10.1080/09540962.2015.1047271.
41–67, 2000 http://www.jstor.org/stable/40398456. 23. Mettler, T., and Rohner, P., An analysis of the factors influencing
5. Melville, N., Kraemer, K. L., and Gurbaxani, V., Information tech- networkability in the healthcare sector. Health Serv. Manag. Res.
nology and organizational performance: An integrative model of IT 22(4):163–169, 2009. https://doi.org/10.1258/hsmr.2009.009004.
business value. MIS Q. 28(2):283–321, 2004 http://www.jstor.org/ 24. Goksen, Y., Cevik, E., and Avunduk, H., A case analysis on the
stable/25148636. focus on the maturity models and information technologies.
6. Aral, S., Brynjolfsson, E., and Van Alstyne, M., Information, tech- Procedia Econ. Financ. 19:208–216, 2015. https://doi.org/10.
nology and information worker productivity task level evidence. 1016/S2212-5671(15)00022-2.
Inf. Syst. Res. 23(3, part 2):849–867, 2012 http://ssrn.com/ 25. Ahern, D., Clouse, A., and Turner, R., CMMI distilled: A practical
abstract=942310. introduction to integrated process improvement. Boston, MA:
7. Han, K., Chang, Y. B., and Hahn, J., Information technology spill- Addison-Wesley, 2003.
over and productivity: The role of information technology intensity 26. Crawford, J., The project management maturity model. Inf. Syst.
and competition. J. Manag. Inf. Syst. 28(1):115–145, 2011. https:// Manag. 23:50–58, 2006. https://doi.org/10.1201/1078.10580530/
doi.org/10.2753/MIS0742-1222280105. 46352.23.4.20060901/95113.7.
J Med Syst (2018) 42: 235 Page 13 of 14 235

27. Mettler, T., Maturity assessment models: A design science research 44. Huang, H. K., PACS is only in the beginning of being used as a
approach. Int. J. Soc. Syst. Sci. 3(1/2):81, 2011. clinical research tool. The 24th international EuroPACS conference
28. Tarhan, A., Turetken, O., and Reijers, H. A., Do mature business pro- Trondheim, Norway, pp. 1–10, 2006.
cesses lead to improved performance? A review of literature for empir- 45. Wetering, R., and Batenburg, R., Towards a theory of PACS de-
ical evidence. In: Proceedings of the 23rd European Conference on ployment: An integrative PACS maturity framework. J. Digit.
Information Systems (ECIS), 26-29 May 2015, Munster, Germany, Imaging 27(3):337–350, 2014.
Association for Information Systems, 1–6, 2015. 46. HIN Advisory Group, Health Information Network (HIN) Maturity
29. NEHTA, Interoperability Maturity Model Version 2.0. National Model. A Discussion Paper. Canadá Health Infoway, 2015. https://
eHealth Transition Authority, Sidney, 2007. https://www. www.infoway-inforoute.ca/en/component/edocman/resources/
digitalhealth.gov.au/implementation-resources/ehealth- reports/2834-health-information-network-hin-maturity-model.
foundations/interoperability-framework. Accessed August 2018. Accessed Sep 2018
30. Grimson, W., Grimson, J., and Hasselbring, W., The SI challenge in 47. Sanders, D., Burton, D. A., and Protti, D., The healthcare analytics
healthcare. Commun. ACM 43:48–55, 2000. https://doi.org/10. adoption model: A framework and roadmap (white paper), 2013.
1145/336460.336474. https://www.healthcatalyst.com/whitepaper/healthcare-analytics-
31. Hasselbring, W., Information system integration. Commun. ACM adoption-model/. Acessed Sep 2018.
43:33–38, 2000. 48. Earthy, J., Usability Maturity Model: Human-Centredness Scale.
32. Scott, J. E., Mobility, business process management, software IE2016 INUSE Deliverable D5.1.4s, 1998. http://www.
sourcing, and maturity model trends: Propositions for the IS idemployee.id.tue.nl/g.w.m.rauterberg/lecturenotes/USability-
Organization of the Future. Inf. Syst. Manag. 24(2):139–145, Maturity-Model%5B1%5D.PDF. Accessed Sep 2018.
2007. https://doi.org/10.1080/10580530701221031. 49. Nielsen, J., Corporate Usability Maturity Stages: 1–4 and 5–8,
33. Tranfield, D., Denyer, D., and Smart, P., Towards a methodology 2006. https://www.nngroup.com/articles/usabilitymaturity-stages-
for developing evidence-informed management knowledge by 1-4/. Accessed Sep 2018.
means of systematic review. Br. J. Manag. 14:207–222, 2003. 50. Schaffer, E., Institutionalization of usability: A step-by-step guide.
34. CMMI-SVC, CMMI® for Services, Version 1.3: Improving pro- Boston: Addison-Wesley, 2004.
cesses for providing better services. Technical Report. CMU/SEI-
51. Staggers, N., and Rodney, M., Promoting usability in organizations
2010-TR-034. ESC-TR-2010-034. The Software Engineering
with a new health usability model: Implications for nursing infor-
Institute. Carnegie Mellon University, 2010.
matics. In proceedings of 11th International Congress on Nursing
35. Garets, D., and Davis, M., Electronic Medical Records versus
Informatics, 23–27 Jun, Montreal, Canadá, 2012.
Electronic Health Records: Yes, there is a difference. Chicago, IL:
52. Mettler, T., and Blondiau, A., HCMM - A Maturity Model for
HIMSS Analytics, 2006, https://www.himss.org/electronic-
Measuring and Assessing the Quality of Cooperation between and
medical-records-vs-electronic-health-records-yes-there-difference-
within Hospitals. in Proceedings of the IEEE Symposium on
himss-analytics. Accessed August 2018.
Computer-Based Medical Systems, 20–22 Jun, Rome, Italy, 2012.
36. Magee, F., IDC MaturityScapes: A Practical Guide to Deriving
53. Brooks, P., El-Gayar, O., and Sarnikar, S., Towards a business in-
Value. IDC white paper, Document #247360. IDC Analyse the
telligence maturity model for healthcare. In proceedings of the 46th
Future, 2016. https://www.idc.com/prodserv/decisionscapes/
Hawaii International Conference on System Sciences, Hawaii,
RESOURCES/ATTACHMENTS/How_to_Use_an_IDC_
USA, 2013.
MaturityScape.pdf. Accessed August 2018.
37. Dunbrack, L. A., Mobile Transformation in Healthcare: 54. Gastaldi, L., Pietrosi, A., Lessanibahria, S. et al., Measuring the matu-
Successfully Competing in a Mobile World, 2013. https://idc- rity of business intelligence in healthcare: Supporting the development
community.com/health/healthcare-transformation/mobile_ of a roadmap toward precision medicine within ISMETT hospital.
transformation_in_healthcare_successfully_competing_in_a_ Technol. Forecast. Soc. Chang. 128:84–103, 2018.
mobi. Accessed August 2018. 55. Clair, C. L., Electronic medical records need more to support
38. Duffy, J., Healthcare Transformation: The Role of IT. Health BMeaningful Use^. Forrester Research Inc, 2010. https://naviant.
Industry Insights, 2006. https://docplayer.net/5588134-Jan-duffy- com/resources/Forrester-Whitepaper-EMR-and-Meaningfu-Use.
research-manager-health-industry-insights-emea.html. Accessed pdf. Accessed Sep 2018.
August 2018. 56. Boer, J., van Houwelingen, J., Adriani, P., and Geerts, A., Game
39. Kenneally, J., Wilson, B., Porter, M., et al., Leveraging IT capabil- maturity model for healthcare: How games will change healthcare,
ities to accelerate business value impact from electronic medical 2013. https://www.compact.nl/articles/game-maturity-model-for-
record adoption. IVI White Paper Series, 2013. http://eprints. healthcare/. Accessed Sep 2018.
maynoothuniversity.ie/6384/ 57. Furukubo, M., Ohuchi, A., and Morimoto, N., Maturity Model of
40. van Dyk, L., and Schutte, C., The telemedicine service maturity Hospital (MMH) for continuous improvement. Proceedings of the
model: A framework for the measurement and improvement of 2nd International Conference on Information Technology for
telemedicine services, telemedicine Ramesh Madhavan and Application (ICITA 2004), 9–11 January, Harbin, China, 2004.
Shahram Khalid, IntechOpen, 2013. https://doi.org/10.5772/56116. 58. Chassin, M., and Loeb, J., High-reliability health care: Getting there
41. Gemmel, P., Vandaele, D., and Tambeur, W., Hospital process ori- from here. Milbank Q. 91(3):459–490, 2013.
entation (HPO): The development of a measurement tool. Total 59. Pinto-Valverde, J., Guardado, M., Ángel, M., Gomez-Martinez, L.,
Qual. Manag. Bus. Excell. 19(12):1207–1217, 2001. https://doi. Corrales-Estrada, M., and Jarquín, J., HDQM2: Healthcare Data
org/10.1080/14783360802351488. Quality Maturity Model. Transactions of the International
42. NHS Connecting for Health, NIMM overview. http://webarchive. Conference on Health Information Technology Advancement, 37,
nationalarchives.gov.uk/20110503153114/http://www. 2013. http://scholarworks.wmich.edu/ichita_transactions/37.
connectingforhealth.nhs.uk/systemsandservices/nimm/overview. Accessed Sep 2018.
Accessed Sep 2018. 60. Smith, L. S., Bradley, R. V., Bichescu, B. C., and Tremblay, M. C.,
43. Liaw, S.-T., Kearns, R., Taggart, J. et al., The informatics IT governance characteristics, electronic medical records sophisti-
capability maturity of integrated primary care centres in cation, and financial performance in U.S. hospitals: An empirical
Australia. Int. J. Med. Inform. 105:89–97, 2017. https://doi. investigation. Decis. Sci. J. 44(3):483–516, 2013. https://doi.org/
org/10.1016/j.ijmedinf.2017.06.002. 10.1111/deci.12019.
235 Page 14 of 14 J Med Syst (2018) 42: 235

61. HIMSS, HIMSS dictionary of healthcare information technology 67. Wetering, R., and Batenburg, R., A PACS maturity model: A system-
terms, acronyms and organizations. 3rd edition. Chicago, IL: atic meta-analytic review on maturation and evolvability of PACS in
HIMSS, 2016. the hospital enterprise. Int. J. Med. Inform. 78:127–140, 2009.
62. Carvalho, J. V., Rocha, A., and Abreu, A., Maturity models of 68. Carvalho, J., Rocha, A., and Abreu, A., HISMM - hospital infor-
healthcare information systems and technologies: A literature re- mation system maturity model: A synthesis. In: Mejia, J. et al.
view. J. Med. Syst. 40(6):131, 2016. https://doi.org/10.1007/ (Eds.), Trends and Applications in Software Engineering,
s10916-016-0486-5. Advances in Intelligent Systems and Computing, 537. Springer
63. van Dyk, L., Schutte, C., and Fortuin, J., A Maturity Model for International Publishing AG, 2017. https://doi.org/10.1007/978-3-
Telemedicine Implementation. In proceeding of eTELEMED 319-48523-2.
2012: The Fourth International Conference on eHealth, 69. HIMSS, Promoting Usability in Health Organizations: Initial Steps
Telemedicine, and Social Medicine January 30–February 4, 2012 and Progress Toward a Healthcare Usability Maturity Model
- Valencia, Spain, 2012. Chicago, IL: Health Information Management Systems Society;
64. CCMM, Continuity of Care Maturity Model (CCMM), 2017. http:// February 2011, 2011.
www.himss.eu/healthcare-providers/continuity-care-maturity- 70. Mettler, T., and Vimarlund, V., Understanding business intelligence in
model-ccmm. Accessed Sep 2018. the context of healthcare. Health Informatics J. 15:254–264, 2009.
65. Škrinjar, R., Štemberger, M., and Hernaus, T., The Impact of 71. Sullivan, J., Rivard, P., Shin, M., and Rosen, A., Applying the high
Business Process Orientation on Organizational Performance. In reliability health care maturity model to assess hospital perfor-
Proceedings of the 2007 Informing Science and IT Education mance: A VA case study. Jt. Comm. J. Qual. Patient Saf. 42(9):
Joint Conference, 2007. 389–399, AP1-AP12, 2016.
66. Holland, M., Piai, S., and Dunbrack, L. A., Healthcare IT Maturity
Model: Western European Hospitals - The Leading Countries (Tech.
Rep. No. HI210231). IDC Health Insights, Framingham, MA, 2008.