FEATURED ARTICLE

Managing Cost of Quality

for Quality 4.0 Using
Software, Systems, and
Data Quality Models
By Daniel Zrymiak
This article shows how a cost of quality approach, already customized for
software and information technology, can be applied toward the deployment and
implementation of Quality 4.0 in an organization. Existing models and quality
characteristics used for software, systems, and data quality can be leveraged to
identify cost categories and to support the creation of a cost of quality system for About the author
Quality 4.0.
Daniel Zrymiak is an
engagement manager at
Cost of Quality Applied to Software Ultranauts, based in Surrey,
British Columbia. He has
While the origins of quality costs and their categories originated in manufacturing
more than two decades of
and industrial engineering, the categories of external failure costs, internal failure
costs, appraisal costs, and prevention costs have been successfully applied to international experience
software and information technology. However, unlike the tangible manufactured in quality and project
product characteristics of the post-war era of the 20th century, software and infor- management, primarily in
mation technology are less tangible. Consequently, the success of such products or information technology,
deliverables is determined during the design and validation processes. operational excellence, and
consulting. Zrymiak is an ASQ
Software and information technology also do not exist as products unto themselves Fellow and has been awarded
but rather were created to perform or automate a particular service. In addition
ASQ’s Feigenbaum and Crosby
to technical deficiencies associated with the software or system, the business
requirements of that service must also be considered. Consequently, quality failure Medals and a Testimonial
can extend beyond functional adequacy to include reliability, responsiveness, and Award. He is a Quality Press
usability. Consider a shopping transaction website that has a three-to-five-minute author and reviewer, and
delay during processing; even though less time is spent than during the alternative committee chair for the ASQ
of travelling to a brick-and-mortar store and waiting in line for the same transac- Edwards Medal. Zrymiak
tion, a delay of this duration would represent an unmet expectation and a service remains active with the Quality
failure of the software.
Management Division as a
member leader and vice-chair
of governance and excellence.
He can be reached at:
13 QUALITY MANAGEMENT DIVISION | SPRING 2021 [email protected].
The Quality Management Forum

The cost of quality model attempts to justify investments in quality appended to finished products or work-in-progress phases.
activities that emphasize planning, prevention, and appraisal. Software and information technology solutions are made using
This is intended to create the resulting deliverable or solution various frameworks and methods, including waterfall, agile,
in a manner that is more stable, predictable, and cost-effective Scrum, prototyping, and evolutionary. These approaches all
than by fixing errors or defects as they emerge in use and are
recognize that a finished solution is the result of releasing a
reported back by the customer base. In the context of software,
the cost of quality categories can be summarized with the verified and validated deliverable, compatible with its operating
following typical activities. environment, suitable for its intended users, and representative of
the original design commitments.

External Failures Appraisal activities must occur throughout all stages or phases
of the design and development projects. The stages may be
These are reported incidents of software or system malfunctions done sequentially (characteristic of waterfall methods) or in a
that prevent the proper use or application of the information
more integrated fashion with repeated iterations leading up to
technology solutions. The impact and extent of the failures (and
an acceptable release. Inspections are supplemented by reviews
subsequent external failure costs) depend on the criticality and
impact of the software. For example, while an online gaming of requirements and designs, audits of development processes,
application isolated to a small percentage of users is of minimal verification that all explicit specifications are integrated, and
concern, software that regulates medical applications or trans- validation of the fitness for use. Appraisal activities continue after
portation arrangements could have catastrophic outcomes. The the product or solution is released into its user environment, and
protection against improper data acquisition or identity theft also they are integrated into the ongoing maintenance activities for
requires additional security measures. If damages are resolved, the life of the product.
there are still marketplace issues, contractual penalties, and
regulatory violations that must be addressed as well. In addition to the direct activities, the supporting environment
and infrastructure must be included within this cost category. For
example, if a software or system was intended to manage the
Internal Failures
concurrent transactions of 25,000 students registering for their
classes, it would be impractical to employ 25,000 testers to
These are found internally, prior to the realization of risks as
business-critical issues. However, the technical remedies are often simulate that quantity of users. A more practical approach would
cumbersome, requiring the proper diagnosis and reconfiguration be to simulate this activity through load, stress, and performance
or redesign of software or its associated systems and operating testing resources, which automatically generate the necessary
environments. The internal failures area is complicated by the test data and user traffic. This has resulted in the formation of a
evolving versions of concurrent or interdependent components, distinct niche in software development dedicated to automated
applications, or infrastructure, which must be constantly software testing.
monitored. Even if the solution was previously determined to be
acceptable, externally directed product or system revisions can
Prevention
also cause disruption and instability. For this reason, all failures
must be diagnosed from a variety of perspectives to determine
Prevention costs are both deliberate and the result of unfortunate
the potential technical and business impact.
experiences learned from product and system failures. While
there are finite ways for a tangible product to fail, software can
malfunction from infinite causes and conditions. This expands
Appraisal the range of prevention activities needed to secure software
and information technology from failures related to functionality,
When this category was first specified for manufactured reliability, usability, portability, efficiency, maintainability, and
products, the types of activities included were primarily product security. Within software design, development, and deployment,
inspections, statistical controls, and process audits. For software such initiatives include research of market and customer pref-
and information technology, this phase cannot simply be erences, requirements review, project planning (that is, scope,

14 QUALITY MANAGEMENT DIVISION | SPRING 2021

The Quality Management Forum

schedule, resources, risks, vendors), systems for configuration be categorized within prevention and appraisal activities, which
management and version control, and training to build these are similar to those currently used for software and information
functional capabilities within the work environment. technology. Attributes of Industry 4.0 are often applied to existing
operations, representing an upgrade to existing software and
As the need for quality is prevalent in software and information information technology. This process can be simplified in a
technology, the costs associated with such activities must also be five-step, high-level process:
monitored and optimized. The cost of quality system has been
leveraged for this purpose and has been used to demonstrate
the positive returns from investments in methods, practices, and Step 1—Analysis
resources that have been shown to reduce the costs of nonconfor-
The project or program is determined based on the business
mance (representing both internal and external failures).
need, the available budget and resources, and the opportunities
for innovation and superior delivery. This analysis needs to
incorporate the context of potential usage situations and the
Quality 4.0: Applying Information and perspectives of all relevant stakeholders, including customers,
Technology for Industry 4.0 Demands vendors, regulatory agencies, partners, and the internal func-
tional areas within the organization. Preventive activities should
work to define the framework of the analysis, while appraisal
Quality 4.0 is a concept that links the practice and application steps should strive to confirm the adequacy of such analysis
of quality techniques to the emerging capabilities associated with relative to the expectations of the proposed solution.
Industry 4.0. Some examples of Industry 4.0 include:

• Additive manufacturing: Enhancing existing systems with 3-D Step 2—Qualification

models and computer controls
The components and vendors are qualified based on the
• Mobile computing: Expanding the devices used by front-line
expected outcomes, the available budget and resources, and the
employees to include smartphones, pads, tablets, and other complexity of the system and solution. The preventive steps are
devices supporting instant interactions and communications represented by the level of detail within documentation (that is,
• Digital manufacturing: Integrating information technology request for proposals), and the appraisal steps should include
the assessment of potential vendors against the established
systems and databases into a holistic end-to-end resource (that
acceptance criteria. Technical solutions often require different
is, enterprise resource planning)
levels of vendor support for technical configurations and design
• Cloud computing: Migration of information technology modifications.
infrastructure to shared services

• Blockchain: Public transaction ledger for tracking and securing Step 3—Development, Verification, and Validation
transaction records
The Industry 4.0 solution is managed through its sub-stages
These examples represent potential augmentations to an or phases of readiness. An increasingly popular framework is
organization’s applications, devices, information management, Scrum, which applies agile methods to rapidly produce incre-
infrastructure, and archiving systems. As all these potential mental iterations until the solution is completed. This approach
solutions would require some form of design, development, infuses preventive activities, which include planning and
configuration, and deployment, the systems currently used for appraisal steps such as inspections and reviews, intermittently
software and data-quality models could be applied. throughout development. One variation, test-driven development,
actually integrates the appraisal tests as requirements against
As with any innovation, risks are present that could result in which development can occur. The increased adoption of test
issues and external failure losses that would be detrimental to automation—and its recurring use for introductory “smoke”
the organization. The countermeasures for such failures would testing or repeated regression testing—represents the adoption

15 QUALITY MANAGEMENT DIVISION | SPRING 2021

The Quality Management Forum

of preventive and appraisal techniques as alternatives to external The quality-in-use model refers to the business needs of the
failures discovered only after deployment to the user community. stakeholders and can be summarized by the following items:

Step 4—Change, Release, Deployment, and Migration • Effectiveness: Accuracy and completeness of usage relative to
the purpose
Any solution proposed for an organization must first be reviewed
for adequacy and suitability, must be released at a time that • Efficiency: Cost and resources needed to fulfill the purpose
minimizes interruptions of existing services and deliverables,
• Satisfaction: Extent that user needs are satisfied by the solution
should be deployed in a controlled manner, and should migrate
all associated services and processes. This step also has to from the intended use
incorporate the complexities associated with training and • Freedom from risk: Mitigation of potential risks, threats, or
adoption. These IT practices (that is, change, release, deploy-
adverse consequences
ment, migration) have distinct preventive and appraisal steps to
ensure that proper reviews, verifications, and validations occur in • Context coverage: The specified conditions and boundaries of
a manner that is commensurate with the complexity and risks of how the system can be used to fulfill the purpose
the newly introduced solutions.
The business risks associated with these items are that use of
these systems can become delayed or disrupted due to their
Step 5—Stabilization and Maintenance
complexity and the resources needed to operate and power
The shifts to new technologies, techniques, and methods will the systems. The failure to manage expectations and deliver the
not be immediately seamless but will need adjustments to desired outcomes over a timely period can increase costs without
address immediate problems and to resolve initial difficulties. delivering on lofty promises of innovation.
In this case, the preventive steps include the development of a
robust knowledge base and responsive approaches to events The potential costs of failure can extend beyond the technical
or incidents. From these rapid responses, the systems can be costs to include the introduction of new threats and adverse
quickly validated and diagnosed, enabling the deployment of
consequences. For example, if user needs are left unmet because
workaround solutions while the root causes are more extensively
the system was overloaded and could not complete transactions,
investigated.
that could result in permanent loss of customer relationships.
Alternatively, if the systems were intended to regulate complex
Quality Characteristics, Business Risks, arrangements such as transportation traffic, a malfunction could
lead to improper routing and costly diversions and accidents.
and Costs of Failure
System/software product quality is associated with the following
Quality characteristics can be identified from a review of two ISO
characteristics:
standards: (1) ISO 25010 Systems and Software Quality Models
and (2) ISO 25012 Data Quality. Unlike tangible products,
which have finite characteristics of size, dimension, functional • Functional suitability: Whether a system is complete, correct,
use, and other evaluation criteria, Quality 4.0 characteristics and appropriate
integrate the principles of information and technology. • Performance efficiency: The capacity, resource uses, and time
consumed
System and Software Quality Models • Compatibility: The interoperability of the system with co-existing
entities
The characteristics of the system and software quality models
include elements of the quality-in-use model and elements • Usability: The ease with which users can access, learn,
associated with system/software product quality. understand, and operate the system

16 QUALITY MANAGEMENT DIVISION | SPRING 2021

The Quality Management Forum

• Reliability: The extent to which systems remain available, • Consistency: Data attributes are not contradictory but coherent
tolerating and recovering from errors with other data

• Security: The protection of confidentiality, authenticity, and • Credibility: Representation of data values as truthful, authentic,
integrity of information and believable by users

• Maintainability: The ability to modify modules, analyze, reuse, • Currentness: The timeliness and correct age for a specific
and test components context of use

• Portability: The ability to install, adapt, and replace systems or

The business risks associated with these items are tied to the
to use them across environments
relevance and importance of the information being handled. If
that information proves deficient or unusable, the value of the
The business risks associated with these items are that systems technology managing that information will be diminished. Since
might not fit the purpose or might introduce new problems and information is used for decision support, any flawed information
complexities that exceed the benefits of the innovation. If a will cause incorrect decisions to be made based on fallacious
system has limited application, it will be an expensive innovation or erroneous indicators or assumptions. If the impacts of those
for a confined segment of the organization. If the user experience decisions are critical to life, security, or property, the risks are
proves difficult, it will affect adoption or prompt bypassing of the infinite and have irreversible consequences.
overly complex or unreliable system to create a “hidden factory”
condition by which work is done without being properly tracked The potential costs of failure can extend beyond those of
or captured. Problems with reliability and security lead to loss of restoring the technical applications and infrastructure to support
business confidence, which can permanently damage existing databases and the resident information. If the data are used for
and future business relationships. particular purposes or specified situations, the context of those
data can be assignable to overall failures in those situations. For
The potential costs of failure can extend beyond the costs of the example, if information about water levels and precipitation was
technical solutions. Introducing the failure costs of the affected inaccurate, and decisions derived from such information resulted
stakeholders and their services can magnify the costs tenfold or in uncontrolled flooding of urban areas, the cost of data failure
even higher. For example, if the systems cause the constituent could be measured by the overall impact to society, traceable
business processes to fail, the external failure costs can cascade back to the data maladies.
from the stakeholders and their client bases, amplifying the losses
and penalties involved. System-dependent data quality characteristics include:

Data Quality • Accessibility: Availability for a specific use or user community

• Compliance: Adherence to standards, conventions, regulations,

The data quality characteristics are segmented by those elements
or rules
that are inherent to the data and those that are dependent on the
system. • Confidentiality: Restriction of data to authorized use, along
with information security
Inherent data quality characteristics include:
• Efficiency: Ability for data to be processed with the available
resources for specific uses
• Accuracy: Representation of data values to true values or
intended attributes • Precision: Exactness of data for a particular use

• Completeness: Possession of all expected attributes and • Traceability: Audit trails of data access, usage, and changes
instances within the data for future reference

17 QUALITY MANAGEMENT DIVISION | SPRING 2021

The Quality Management Forum

• Understandability: Ability of the data to be read and under- These are all traceable back to the software, systems, and data
stood impacted by the Quality 4.0 innovations and upgrades applied.
• Availability: Data can be retrieved by authorized users within
the context of a purpose When the potential costs and impacts are quantified, the proba-
bilities of such events could accommodate a realistic risk-based
• Portability: Data are installed, replaced, or moved between estimate. For example, a 0.1% chance of a catastrophic failure
systems remaining intact costing $1 million would translate into an estimated risk costing
• Recoverability: In the event of a failure, data operations can be $1,000. This figure could be traded against the prevention and
maintained and preserved appraisal steps needed to mitigate the risk, so the costs are
planned and predictable, and the risks are stabilized.

The business risks associated with these items are that systems Optimization would come from consolidating the potential
will manage the data in a manner that diminishes their integrity countermeasures taken for planning, project management,
or usability. The flawed systems may cause data to be lost, change controls, reviews, verifications that specifications were
corrupted, or disorganized such that their value to their constitu- met, validations of fitness for intended uses in the required envi-
ent stakeholders and their processes is invalidated and nullified. ronments, and other types of inspections and audits. By consoli-
dating these practices into workable and repeatable systems, the
The potential costs of failure can include the ongoing cost existing risks are mitigated, and the robustness increases beyond
burdens of managing and storing data that consume time and problem resolution to innovations and breakthroughs.
resources without providing commensurate benefit to their
intended users or to the interoperating applications and systems. Quality 4.0 endeavors already have a prescribed business case
The discovery and diagnosis of these problems create additional to justify their adoption. However, the presence of external and
work, particularly unplanned recovery from unanticipated internal failures forces the projects to adopt additional rework
disasters. and delays. The absence of failures increases the “purity” of
returns, and the predictability of future innovations for stakehold-
Business continuity practices have been established specifically ers and their affected activities.
to address the potential failure conditions. The alternative to
planned business continuity is the wasteful and time-consuming
rework involved to recreate and replicate any information that Conclusion: Key Outcomes
was lost or corrupted. Where this cannot be done automatically
or internally, customers or users may then be required to submit The cost of quality protocol can be applied to Quality 4.0. The
redundant information, causing distress and frustration with deployment can be elaborated using the same models and
respect to customer engagement. practices as are currently applied to software, systems, and data.
Quality 4.0, comprising primarily software and information
technology, consists of intangible deliverables that are provided
Managing Quality Costs and Optimizing to stakeholders, that are applied to processes and solutions, and
that require sufficient quality cost controls to mitigate losses from
Returns on Investment potential internal and external failures.

Having reviewed the breadth of areas that potentially require

quality safeguards, the projected costs of external failures would
include the expenses to design, develop, deploy, and maintain
Quality 4.0 deliverables, plus potential costs from losses to
stakeholders and their affected processes and customer solutions.

18 QUALITY MANAGEMENT DIVISION | SPRING 2021