Introduction to CMMI Appraisals

An appraisal is an activity that helps you to identify strengths and weaknesses of your organizations
processes and to examine how closely the processes relate to CMMI (or People CMM) best practices.
Preparing for an appraisal helps your organization to do any of the following:
> Plan an improvement strategy for your organization.
> Determine the CMMI levels that represent how well your organizations processes conform to CMMI
> Mitigate risks for product and service acquisition, development, delivery, and monitoring.
> Demonstrate to customers and business partners the soundness of your processes by having your
appraisal results available on the Published Appraisal Results (PARS) site.
By knowing the appraisal results of prospective suppliers and business partners, you can make better
decisions about creating business relationships with them. The appraisal results of competitors help you to
understand them and compete more effectively.
The Standard CMMI Appraisal Method for Process Improvement (SCAMPI) is the official CMMI (and People
CMM) appraisal method used to evaluate organizations processes and provide ratings. This method is
documented in the SCAMPI Method Definition Document, which is one of the appraisal reference documents
used during an appraisal.
The SCAMPI includes three appraisal classes. SCAMPI A appraisals are officially recognized appraisals that
result in benchmark quality ratings (e.g., maturity levels). SCAMPI B & C appraisals are less rigorous
appraisals designed to provide information for process improvement or the status of process improvement
work.
Levels are used in CMMI solutions to describe evolutionary paths recommended for organizations that wish
to improve their processes used to acquire, develop, and deliver products and services. Levels can also be
the outcome of the rating activity in SCAMPI appraisals.
CMMI for Acquisition, CMMI for Development, and CMMI for Services all support both capability levels and
maturity levels. The People CMM supports maturity levels only.
Capability levels enable your organization to focus its process improvement efforts process area by process
area from capability level 0 to capability level 3. Once you select a process area, you decide how much you
want to improve the processes associated with that process area (i.e., select the appropriate capability level).
The achievement of capability levels is illustrated in a capability profile.
Maturity levels provide a staging of processes for improvement across your organization from maturity level 1
to maturity level 5. This improvement involves achieving the goals of the process areas at each maturity level.
Process areas can be grouped by maturity level, indicating which process areas to implement to achieve
each maturity level. Maturity levels are illustrated by a single number (e.g., maturity level 3).
To reach a particular level, an organization satisfies all of the goals of the process area or set of process
areas that are targeted for improvement, regardless of whether it is a capability or a maturity level. Both are
valid ways to improve your processes to achieve business objectives; and both provide the same essential
content and use the same model components.
Your process improvement goals should always be based on your business objective. Experience has shown
that organizations benefit from achieving a level only when the focus of improvement is on shared higherlevel objectives, not a focus on achieving the level itself. When the focus is on achieving business objectives,
the levels result naturally.

What Is SCAMPI?
The Standard Capability Maturity Model Integration (CMMI) Method for Process Improvement (SCAMPI)
is designed to provide a well-defined set of methodologies for conducting appraisals relative to CMMI
models. Each SCAMPI method satisfies all of the appraisal requirements for CMMI (ARC) for the
appropriate Class A, B, or C appraisal method. SCAMPI A satisfies all of the ARC requirements for a
Class A benchmark appraisal. Although designed for conducting appraisals against CMMI-based
reference models, the SCAMPI method can also be applied for conducting appraisals against the People
CMM and other reference models.
SCAMPI A appraisal sponsors are encouraged to consider the SCAMPI B or SCAMPI C appraisal
methods, which may produce acceptable results where ratings are not required, with significantly less
investment of cost and resources.
Key differentiating attributes for appraisal classes include

The degree of confidence in the appraisal outcomes

The generation of ratings
Appraisal cost and duration

A SCAMPI enables a sponsor to

Gain insight into an organizations capability by identifying the strengths and weaknesses of its
current processes relative to appraisal reference model(s)
Focus on improvements (correct weaknesses that generate business risks) that are most
beneficial to the organization given its current level of process implementation
Identify risks relative to achieving capability or maturity targets

As a Class A appraisal method, SCAMPI A is appropriate for generating ratings as benchmarks to

compare maturity or capability levels across organizations. As a benchmarking method, the emphasis is
on a rigorous method capable of achieving high accuracy and reliability of appraisal result through the
collection of objective evidence from multiple sources. Sponsors who want to compare an organizations
process improvement achievements with other organizations in the industry may have a maturity level
determined as part of the appraisal process. Organizational maturity level or capability level ratings
resulting from a SCAMPI A are valid for up to three years.
Decisions made on the basis of maturity level ratings are only valid if the ratings are based on known
criteria. Consequently, contextual informationorganizational unit, organizational scope, reference model
scope, appraisal method type, the identity of the appraisal team leader and the teamare items for which
criteria and guidance are provided within the method to ensure a consistent interpretation within the
community. Benchmarking can only be valid when there is a consistent basis for establishing the
benchmarks.
The SCAMPI B appraisal method complies with a subset of ARC requirements. Several requirements of
the SCAMPI A method are optional for the SCAMPI B method. Two types of objective evidence are
required for both SCAMPI A and B methods. The SCAMPI B method does not produce ratings. These
types of appraisals are recommended for initial assessments in organizations that are just beginning to
use CMMI models for process improvement activities. They also provide a cost-effective means for
performing interim assessments and/or capability evaluations between SCAMPI A appraisals.
SCAMPI C appraisal method complies with a subset of ARC requirements for the SCAMPI B method.
Only one of the two types of objective evidence required for SCAMPI A and SCAMPI B methods is
required for the SCAMPI C method. The SCAMPI C method does not produce ratings. Corroboration is
also optional for the SCAMPI C method. These types of appraisals would most likely be used when the
need for a quick look arises or for periodic self-assessments by projects and organizational support
groups.
The CMMI Institute maintains industry aggregates for SCAMPI appraisal results. These data are
reported in industry maturity profiles gathered from organizations that have performed appraisals since
1987. The profile is based on appraisal data provided by CMMI Institute-trained professionals, and is
updated twice annually.
Excerpt from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.3a: Method Definition Document. March
2011. Pages 7-11.
http://cmmiinstitute.com/resource/standard-cmmi-appraisal-method-process-improvement-scampi-b-c-version-1-3a-method-definition-document/

The CMMI Institute supports the SCAMPI methods and operates a certification program for SCAMPI lead
appraisers. Additional details can be found on the CMMI Institute web site at
http://certification.cmmiinstitute.com/.

Core Concepts and Approach

SCAMPI relies on an aggregation of information that is collected via defined types of objective evidence.
The objective evidence feeds an information-processing engine whose parts are made up of a series of
data transformations. The appraisal team observes, hears, or reads information that is transformed into
notes and then into model gaps, and then into findings. These findings are typically validated by the
organizational unit before they become final findings. The critical concept is that these transformations
are applied to data reflecting the enacted processes in the organizational unit and the appraisal reference
model, and this collection of data forms the basis for appraisal results.
Planning is critical to the execution of a SCAMPI. All phase and process activities briefly discussed below
derive from a well-articulated plan developed by the appraisal team leader in concert with members of the
appraised organization and the appraisal sponsor.

SCAMPI Methodology
SCAMPI consists of three phases and several essential processes, as was shown in Table 2.
Each phase is described in detail below.
Phase 1: Plan and Prepare for Appraisal
Appraisal planning starts with understanding the sponsors objectives, requirements, and constraints. All
other planning, preparation, execution, and reporting of results proceed from this initial activity. Because
of the significant investment and logistical planning involved, considerable iteration and refinement of
planning activities should be expected in phase 1. With each subsequent phase, the amount of iteration
will decrease as data are collected, analyzed, refined, and translated into findings of significance relative
to the model.
Experienced and trained personnel perform a SCAMPI over a period of time negotiated by the sponsor
and the appraisal team leader. The scope of the organization to be appraised, as well as the scope of the
appraisal reference model (process areas), must be defined and agreed upon. The scope of the
organization and model provides the basis on which to estimate personnel time commitments, logistical
costs (e.g., travel), and overall costs to the appraised organization and to the sponsoring organization.
During the appraisal, the appraisal team verifies and validates the objective evidence provided by the
appraised organization to identify strengths and weaknesses relative to the appraisal reference model.
Objective evidence consists of artifacts and/or affirmations (refer to glossary for definitions, page xlvi)
used as indicators for implementation and institutionalization of model practices or model components.
Before the Conduct Appraisal phase begins, members of the appraised organization typically collect and
organize documented objective evidence, using defined data collection strategies based on the extent of
artifacts available within the organization and aligned with the appraisal reference model.
Advance preparation by both the appraisal team and the appraised organization is key to the most
efficient execution of the method. Analysis of preliminary documented objective evidence provided by
the appraised organization plays an important role in preparing for appraisal execution. If substantial
data are missing at this point, subsequent appraisal activities can be delayed or even cancelled if the
judgment is made that continuing appraisal activities will not be sufficient to make up for the deficiency
given the resources available.

Excerpt from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.3a: Method Definition Document. March
2011. Pages 7-11.
http://cmmiinstitute.com/resource/standard-cmmi-appraisal-method-process-improvement-scampi-b-c-version-1-3a-method-definition-document/

The collection of documented objective evidence by the appraised organization in advance of the
appraisal can help to improve appraisal team efficiency and offer several other benefits to the
organization including the following:
Improved accuracy in appraisal results delivered by external appraisal teams (i.e., clear
understanding of implemented processes, strengths, and weaknesses)
Detailed understanding of how each part of the organization participating in the appraisal has
implemented model practices, and the degree of compliance and tailoring of organizational
standard processes
The establishment of residual appraisal assets that can be reused on subsequent appraisals,
minimizing the effort necessary for preparation
However, the effort to collect, organize, and review large amounts of objective evidence in advance of the
appraisal can be a large cost to appraised organizations, and can lead to diminishing returns if not done
efficiently. Incremental data collection strategies with specific data requests can help to mitigate the risks
of inefficiently using the organizations resources on collecting data that is not appropriate or useful. A
data collection plan, developed by the appraisal team leader in conjunction with the appraised
organization, can help make explicit the choices on how much data collection effort to distribute between
the organization and the appraisal team.
Phase 2: Conduct Appraisal
In phase 2, the appraisal team focuses on collecting data from the appraised organization to judge the
extent to which the model is implemented. Integral to this approach is the concept of coverage, which
implies two things: first, the collection of sufficient data for each model component within the model scope
selected by the sponsor, and second, obtaining a representative sample of ongoing processes.
The organizational scope, determined in phase 1, is based on the understanding of unique process
implementations within the appraisal scope, with the goal of optimizing the effort for collection and
analysis of objective evidence. This means collecting data and information on all the appraisal reference
model components in the appraisal scope, and across sampled process instantiations within the
organizational unit being appraised. The data collection plan developed in phase 1 undergoes continuous
iteration and refinement until sufficient coverage is achieved.
Upon determining that sufficient coverage of the appraisal reference model and organizational unit has
been obtained, appraisal results may be generated.
Phase 3: Report Results
In phase 3, the appraisal team provides the appraisal results to the appraisal sponsor and the
organization. These artifacts become part of the appraisal record, which becomes protected data in
accordance with the appraisal disclosure statement. The level of protection and the plan for the
disposition of appraisal materials and data are determined in phase 1 in collaboration with the sponsor. A
completed appraisal data package, which includes a subset of the contents of the appraisal record, is
provided to the CMMI Institute. The CMMI Institute adds the appraisal data to confidential databases, and
provides overall profiles of the community on a periodic basis.

Excerpt from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.3a: Method Definition Document. March
2011. Pages 7-11.
http://cmmiinstitute.com/resource/standard-cmmi-appraisal-method-process-improvement-scampi-b-c-version-1-3a-method-definition-document/

SCAMPI Tailoring
Successful application of SCAMPI relies on adjusting the parameters of the method to the needs of the
organization and to the objectives and constraints of the sponsors organization.
The sponsors objectives largely influence tailoring decisions. The reference model scope and
representation (staged or continuous), the size of the organizational unit, the selected parts of the
organization, the size of the appraisal team, and the number of interviews greatly influence things such as
preparation time, time on site, and monetary costs, and so are also major factors when choosing tailoring
options. All tailoring decisions must be documented in the appraisal plan.
Appropriate tailoring of the appraisal method provides flexibility to efficiently adapt the appraisal to the
needs of the sponsor. The appraisal team leader is responsible for ensuring that the requirements of the
method are satisfied. Tailoring the method too severely could result in failure to satisfy method
requirements, the inability to obtain sufficient data for generation of appraisal results, or failure to meet
the criteria necessary for recognition as a valid SCAMPI appraisal.

Time Frame and Personnel Requirements

A requirement of the SCAMPI A method is that the Conduct Appraisal phase must be completed within 90
days. SCAMPI A appraisal results are valid for up to 3 years. Since neither SCAMPI B nor SCAMPI C
produce ratings, no specific time constraints are imposed on the conduct or validity of appraisal results for
SCAMPI B or C methods.
Personnel needed to participate in activities or perform tasks in a SCAMPI appraisal include the sponsor,
the appraisal team leader, the appraisal coordinator, selected participants, and appraisal team members.
Their time commitments will vary widely depending on the specific parameters of the appraisal (e.g.,
organizational scope) and their role.
Much of the effort spent by an organization preparing for an appraisal is for the collection and review of
objective evidence; these costs can vary widely based on the data collection strategy and acceptable
level of risk. Excluding data collection costs, appraisal participants can typically expect to spend one to
three hours each to participate in interviews and attend validation sessions, plus one to three hours each
for presentations. On the other extreme, the appraisal coordinator may spend weeks of full-time effort
helping the team and the organization to prepare for and conduct the appraisal. Appraisal team leaders
should engage appraisal sponsors on effort estimates and the set of tailoring options to be used in
conducting a SCAMPI appraisal.

Excerpt from Standard CMMI Appraisal Method for Process Improvement (SCAMPISM) A, Version 1.3a: Method Definition Document. March
2011. Pages 7-11.
http://cmmiinstitute.com/resource/standard-cmmi-appraisal-method-process-improvement-scampi-b-c-version-1-3a-method-definition-document/