What is AI risk?

AI risk management has become a best practice among enterprises in order to measure and mitigate
risk to meet organizational requirements. To fully understand this, we must first define AI risk.

AI risk can be expressed as a probabilistic expectation:

AI risk = (likelihood of an AI model error or exploit) x (its potential effect)

This definition is instructive because it provides a recipe for action. AI errors and vulnerabilities are
many, occur frequently, and vary across model tasks and modalities. Examples include arbitrary code
execution, data poisoning, prompt injection, model extraction, hallucinations, data drift, unexpected
behaviour, bias predictions, and toxic output.

The effects of a model error depend largely on the use case. They can be financial, legal, or
reputational. More importantly, they can have devastating consequences for users, such as leaked
personal information, denied healthcare coverage, or declined loan approval decisions.

How is AI risk different from traditional software risk?

AI models are not just a function of the underlying “code”; they are fundamentally also a function of
the data. In traditional software functions, one defines rigorous abstractions rooted in procedural
software logic in order to transform an input into a given output. Yet the very advantage of AI is that
it can “learn” the logic from a corpus of data, rather than the user explicitly defining it. Because the
user cannot directly observe this logic, this makes the burden of testing much harder on the user.
Simply testing on a few data points and edge cases is no longer enough to assert “correctness”. The
user must now rigorously test the performance of the model with evaluation datasets, and assert
that the model can also make correct predictions on a potentially infinite set of possible data
distributions.

A corollary is that the metrics used to evaluate the performance of a AI model are different from
those used to test software. This is because AI models are approximate learners on noisy data; in
most practical settings, it is impossible for models to be 100% correct on the entire data distribution.
For example, accuracy is a common metric for evaluating the performance of a classification model,
whereas software is often tested for functionality and usability. In addition, simply testing top-level
metrics on a dataset is not enough. There are many other dimensions in which AI models can fail.

AI models also come in a wide array of modalities/tasks, ranging from binary classification on tabular
data to language-based responses from generative AI. Each modality and tasks come with their own
unique failure modes and testing challenges in the dimensions mentioned above. Moreover, since AI
involves automating critical decisions, there are test that we would like to run that pertain to other
aspects of integrity of the system. This includes testing against bias and fairness, as well as abuses to
sensitive information that may be encoded in the model and data.

AI is presents unique challenges that necessitate a new paradigm to mitigate risk.

What is AI risk management?

AI risk management refers to a suite of tools and practices deployed to proactively protect
organizations and end users from the distinctive risks of AI. It involves measuring the risks, and
implementing solutions to minimize them. As per the definition of AI risk, it can be minimized by
reducing the likelihood of a failure or by reducing the impact severity. Although it may resemble
some software engineering best practices, AI risk management is entirely different.
According to NIST, as defined in their AI Risk Management Framework:

AI risk management is a key component of responsible development and use of AI systems.

Responsible AI practices can help align the decisions about AI system design, development, and uses
with intended aim and values. Core concepts in responsible AI emphasize human centricity, social
responsibility, and sustainability. AI risk management can drive responsible uses and practices by
prompting organizations and their internal teams who design, develop, and deploy AI to think more
critically about context and potential or unexpected negative and positive impacts. Understanding
and managing the risks of AI systems will help to enhance trustworthiness, and in turn, cultivate
public trust.

Why do companies need AI risk management?

1. Security Risk

Security risk refers to AI system vulnerabilities that may be exploited maliciously or inadvertently
added by end users to generate an unintended output. This can include attacks on the model, data,
or underlying software. Proprietary, commercial, and open-source models may be susceptible to a
variety of threats including supply chain risk, data poisoning, prompt injection, PII extraction, and
model theft.

2. Ethical Risk

Ethical risk stems from model behavior that violates norms, laws, regulations, or other governance
standards. This may be present in the training data or be the result of production data over time.
Examples include bias predictions, toxic outputs, exclusiveness, and prejudice responses.

3. Operational Risk

Operational risk is the result of deviations in model predictions. This can be the result of data drift,
hallucination, corrupt data, corner case inputs, and broken data pipelines. Silent model failures are
especially difficult to detect, as they don’t crash the model outright and have subtle effects on
downstream metrics.

AI risk management as an industry standard

AI risk is the subject of existing and forthcoming AI standards and regulation. The landscape is
comprised of voluntary frameworks, guidelines, and legislation. Many companies are adopting AI risk
management now in order to get ahead of anticipated legislation and additional regulation.

The most notable examples include:

The U.S. Congress recognized the need for guidance in AI risk management and mandated NIST
(National Institute of Standards and Technology) to create a framework to do just that. In January,
2023 they released the first version of the NIST AI Risk Management Framework.

The Biden-Harris Administration secured voluntary commitments from 15 large AI companies to

adhere to specific AI risk management measures. This agreement serves to accomplish two goals:
increase the public’s confidence in AI services developed by “big tech” and serve as a benchmark for
what enterprises should require from their own AI systems, including the vendors in their AI stack.
The EU AI Act is a first of its kind legislative initiative aimed at regulating AI on a risk-based
approach. The text is very prescriptive for high-risk AI providers, notably in risk management, data
governance, technical documentation, and record keeping.

Automating AI risk management

Manual evaluation can be slow and expensive. Large enterprises spend millions of dollars to
manually discover errors in their AI and engineer mitigations. Using tools to automate AI risk
management can turn passive frameworks into active practices. This automation helps to proactively
measure and mitigate risk, giving companies the confidence to deploy AI at scale. There are many
benefits, including:

Real-time validation

It should be expected that all models will receive malicious inputs and generate undesired outputs;
generative models are especially vulnerable since outputs are passed to users immediately. Real-
time validation of inputs and outputs is an essential form of automation that is required to protect
production models.

Comprehensive testing

The non-deterministic nature of AI models make it difficult to predict failures. Automated,

comprehensive testing is far superior to manual, ad hoc testing of models and data. By running
hundreds of tests and algorithmically-generated red teaming attacks, you can ensure your models
are rigorously validated. This also helps enforce a uniform risk standard across the company rather
than rely on the subjectivity of any individual or team.

Resource efficiency

Manual validation of models is slow and expensive. It can take upwards of 80 hours to test each
model and map the results to an AI risk management framework, as opposed to a fraction of that
with automated validation. This process needs to be repeated each time a new version of a model is
released, which happens frequently with commercial and open-source models. An automated
process also removes the validation burden from data scientists, who are typically not experts in
governance and compliance.
Implementing ISO Standards for Quality Management of AI Systems (Written by: Mike Curcurito):

Key Takeaways

 Effective management and governance are essential for maximizing the benefits of AI, while
minimizing risks.
 ISO 27001 and 42001 standards provide a framework for organizations navigating the
complexity of AI management systems.
 By utilizing these standards, organizations can build trustworthy AI management systems to
address security, privacy, and quality objectives.

As we all know, artificial intelligence (AI) has emerged and continues to be the hot topic,
revolutionizing various sectors from healthcare to finance. However, the development and use of AI
systems pose unique challenges that require robust management and governance. This is where ISO
27001 and ISO 42001 standards come into play.

What is ISO 27001 & ISO 42001?

ISO 27001 is a globally recognized standard that provides a framework for establishing,
implementing, maintaining, and continually improving an information security management system
(ISMS). It is designed to help organizations manage their security practices in one place, consistently
and cost-effectively.

On the other hand, ISO 42001 is a standard specifically designed for the responsible development,
use, and management of AI systems. It provides guidance on incorporating AI management systems
(AIMS) with general or industry-specific management standards and emphasizes the importance of
structured frameworks for AIMS.

As AI systems become increasingly complex, the need for structured frameworks to manage these
systems is vital. These frameworks not only ensure the responsible use and development of AI
systems, but also helps organizations meet their security, privacy, and quality objectives.

The integration of ISO 27001 and ISO 42001 provides a comprehensive approach to AI governance. It
allows organizations to address security objectives in a systematic way, comply with privacy
obligations, and ensure the quality of AI systems. These standards facilitate business across
organizations and inspire customer confidence in products or services involving AI technologies. In
the following sections, we will explore deeper into how implementing these ISO standards can
effectively manage AI systems, providing a secure, reliable, and responsible AI environment.

Scope of AI Systems

AI systems encompass a wide range of technologies, from machine learning algorithms to

autonomous robotics. The scope of an AI system refers to the boundaries and applicability of the
system within an organization. It is essential to define this scope to ensure that the AI system aligns
with the organization’s strategic direction and business processes.

However, the complexity and diversity of AI technologies make it challenging to establish a clear
scope. AI systems can range from simple rule-based systems to advanced neural networks capable of
deep learning. Each of these systems have different requirements, risks, and impacts, requiring a
tailored approach to their management.
The importance of establishing a clear scope for AI systems cannot be overstated. A well-defined
scope provides a roadmap for the implementation, operation, and monitoring of the AI system. It
helps in identifying the resources needed, the potential risks, and the measures required to mitigate
these risks. Additionally, it aids in setting clear objectives for the AI system, ensuring that these
objectives align with the organization’s overall goals.

The Role of ISO 27001 & ISO 42001

ISO 27001 and ISO 42001 standards provide a structured framework for defining the scope of AI
systems. These standards guide organizations in determining the boundaries and applicability of
their AIMS. They consider both internal and external issues, as well as the specific requirements of
the AI system.

By utilizing ISO standards, organizations can ensure that their AI systems are managed in a
systematic and responsible manner. This not only enhances the effectiveness of the AI system but
also promotes its responsible use and development. The standards apply to all organizations,
regardless of their size, type, or nature, as long as they provide or use products/services that involve
AI systems.

The following five components will help organizations to determine the scope of their AIMS:

 Determine Boundaries and Applicability: Identify specific AI technologies, processes, and

systems covered under the AIMS. For example, include chatbots for customer service,
exclude AI used in data analysis.
 Consider External and Internal Issues: Account for external factors like regulatory
requirements, and internal factors such as organizational capabilities.
 Address Requirements: Identify relevant interested parties’ requirements, including legal,
customer, and business needs.
 Document the Scope: Formalize and make the AIMS scope accessible to stakeholders. For
example, publishing the scope on internal intranet for easy access.
 Aligning with Organizational Activities: Ensure the AIMS scope aligns with organizational
activities and objectives.

Organization & Interested Parties in the AI Management Process

Organizational Roles

An organizational role (CISO, IT Manager, etc.) refers to a defined responsibility or set of

responsibilities within an organization. These roles are typically approved by top management and
are a focal point in the governance of AI systems. They act as the primary entity responsible for the
development, deployment, and management of these systems. Their role extends to the
identification of stakeholders, which is a crucial step in the AI management process.

Stakeholders

Stakeholders in the AI ecosystem can range from internal teams, such as IT and data science
departments, to external entities like customers, regulatory bodies, and the public. Recognizing
these stakeholders is essential as they can significantly influence the development and deployment
of AI systems. Their needs, expectations, and concerns can shape the direction of AI initiatives, and
refine AI models and algorithms, stressing the importance of having a structured framework.

How ISO 27001 & ISO 42001 Standards Come Into Play
ISO 27001 and ISO 42001 standards provide a comprehensive set of guidelines for implementing an
AI management system, addressing key areas such as security, privacy, and quality. They facilitate
the management of stakeholder concerns by providing a systematic approach to identifying risks,
implementing controls, and ensuring continuous improvement.

Key components in ISO 42001, such as organization, interested party, and top management assist in
identifying the appropriate parties. However, an organization must analyze its structure, operations,
and stakeholders, mapping out hierarchies, decision-makers, policies, and objectives. Considering
the scope of AI management systems and interested parties’ requirements is essential.

By adhering to these standards, organizations can demonstrate their commitment to responsible AI

use, enhancing stakeholder trust and confidence. These standards serve as a valuable tool for
organizations to effectively manage their AI systems, considering the interests of all relevant roles
and stakeholders.

Documented AIMS Policy

Documented policies in the management of AI systems are paramount. These policies serve as a
roadmap, guiding the management of AI systems in a manner that aligns with the organization’s
objectives, and complies with legal and regulatory requirements. They provide a structured
framework that promotes responsible and ethical use of AI, ensuring that it is used to enhance
operations and decision-making without compromising security, privacy, and other critical aspects of
the business.

However, the creation and maintenance of these policies require a formalized approach. ISO 27001
and 42001 standards provide comprehensive guidelines for establishing, implementing, maintaining,
and continually improving an ISMS within the context of the organization. They emphasize the
necessity of top management’s commitment, the establishment of AI policy, and the assignment of
roles and responsibilities, among other things.

For instance, ISO 42001 mandates that top management establish an AI policy that is appropriate to
the purpose of the organization, provides a framework for setting AI objectives, and includes a
commitment to meet applicable requirements and improvement of the AIMS. It also requires that
this policy be documented, communicated within the organization, and made available to interested
parties as appropriate.

Additionally, ISO 42001 provides controls and implementation guidance for establishing an AI policy
and defining and allocating roles and responsibilities. It also requires the review of the AI policy at
planned intervals or as needed to ensure its continuing suitability, adequacy, and effectiveness. By
adhering to these ISO standards, organizations can ensure that their AI systems are managed in a
manner that is systematic, consistent, and compliant with best practices. This enhances their
effectiveness and efficiency by reducing risks and promoting trust among stakeholders.

AI Policy Components

Without a well-defined policy, organizations may struggle to effectively manage risks, leading to
potential legal, financial, and reputational damage. To limit these risks, management should include
the following in their AI Policy:

Alignment with Mission and Vision: The AI policy should articulate how AI supports the
organization’s core values and strategic objectives.
Communication and Accessibility: The policy should be effectively communicated to all relevant
stakeholders and readily accessible. Clear communication ensures understanding and compliance.

Management of Diverse AI Systems: Recognize that different AI systems may be in use across the
organization. The policy should address the management and integration of these diverse systems
cohesively.

Authorization and Usage Guidelines: Define who is authorized to use AI systems within the
organization and under what circumstances. Establish clear guidelines for appropriate and
inappropriate usage.

Interaction with Other Policies: Recognize how AI policy intersects with other organizational
policies, such as data privacy, cybersecurity, third-party management, incident response, and ethics.

Reporting Mechanisms and Feedback Loops: Establish documented processes for reporting AI-
related incidents, feedback mechanisms for continuous improvement, and avenues for addressing
concerns.

Data Resources Management: Address the sourcing, storage, and usage of data within AI systems,
ensuring compliance with data protection regulations and ethical considerations.

AI System Lifecycle Management: Outline the phases of AI system development, deployment,

operation, maintenance, and retirement by incorporating risk management and continual
improvement practices.

Use of AI Systems in Operations: Define how AI systems are integrated into operational processes,
ensuring efficiency, reliability, and adherence to organizational goals.

Management of Third-Party and Customer Relationships: Address the use of AI in interactions with
third-party vendors, customers, and stakeholders, emphasizing transparency, fairness, and trust.
There should also be a documented policy that should be acknowledged by third parties or
customers on what appropriate and inappropriate use is, and the intended use of the AI systems.

By addressing these additional components in the AI policy, organizations can increase trust,
transparency, and accountability in their AI initiatives, while mitigating risks and ensuring
compliance with legal and ethical standards. The policy should be reviewed and approved by top
management on a periodic basis.

Objectives of AI Systems

The key objectives of AI systems are multi-layered and can vary greatly depending on the specific
application or sector. These objectives can range from diagnosing and treating illnesses to
augmenting security measures, such as predicting threat detection and prevention of security
attacks. Identifying and documenting these objectives is a necessary step in the responsible design
and development of AI systems. Management should consider both the positive and negative
outcomes when using AI systems, such as health and safety-related scenarios.

ISO 27001 is key in most contexts where security is a primary objective. The way an organization
pursues security objectives depends on its context and its own policies. If an organization identifies
the need to implement an AI management system and to address security objectives in a thorough
and systematic way, it can implement an ISMS in conformity with ISO 27001.

ISO 42001 provides requirements and guidance from an AI technology-specific view. It emphasizes
the need for the responsible development and use of an AI system, taking into account the AI-
specific considerations and the system as a whole. The ISO standards provide a structured
framework that aids in setting and achieving objectives for AI systems. They ensure that measures
are integrated into the various stages of AI system development, from requirements specification,
data acquisition, data conditioning, model training, verification, and validation. This integration of
ISO standards with AI management systems is essential for the responsible development and use of
AI systems, ultimately leading to the attainment of the identified primary goals and objectives in AI
implementation.

Setting AI Objectives with SMART Metrics

To set the AI objectives, management needs to understand the organizational goals and align the AI
objectives with the overall mission. If an organization has set a business objective to enhance
customer experience, then the AI objective might focus on implementing chatbots for customer
support to improve response times and satisfaction rates.

When identifying all objectives and aligning them accordingly, management should then consider
setting SMART metrics to track the progress and success of the AI systems. SMART metrics can be
defined as follows for AI systems:

Specific: Goals and objectives should be clear, precise, and well-defined. They should focus on a
specific aspect or outcome of the AI system, avoiding ambiguity or vagueness.

Measurable: Objectives should be quantifiable and measurable, allowing progress to be tracked and
evaluated over time. This involves defining concrete metrics or key performance indicators (KPIs)
that can be used to assess the success of the AI initiative.

Achievable: Objectives should be realistic and attainable within the organization’s resources,
capabilities, and constraints. Factors such as technology readiness, available expertise, and
budgeting should be considered.

Relevant: Objectives should be aligned with the organization’s overall goals, strategic priorities, and
mission. They should contribute directly to the organization’s success and address key challenges or
opportunities relevant to the business.

Time-bound: Objectives should have a defined timeframe or deadline for achievement. This creates
a sense of urgency and helps prioritize efforts. It also enables progress to be monitored effectively
and ensures that resources are allocated efficiently.

By setting SMART metrics for implementing AI objectives, organizations can effectively track
progress, measure success, and ensure that efforts are focused on achieving tangible
improvements in their established goals. After SMART metrics are applied, then the objectives
should factor in the expectations and needs of various stakeholders, including customers,
employees, investors, and regulatory bodies.

Once the objectives are set and aligned with stakeholders’ top priorities, then management should
focus on allocating resources and defining responsibilities. The organization should assign a leader or
leaders of the objective who will then determine the resources (budget, skilled personnel, etc.)
required to achieve the AI objectives.

An example of this could be allocating funds for acquiring AI software licenses, hiring data
scientists, and training staff on AI technologies. The application of the objectives should be
monitored and adjusted as needed. The assigned leader should be responsible to continuously
monitor progress towards achieving AI objectives, and be prepared to adjust based on feedback and
changing business priorities.

Risk & AI System Impact Assessment

A risk assessment and an AI system impact assessment (AISIA) are top priority in managing AI
systems. The risk assessment involves identifying potential threats and vulnerabilities that could
compromise the integrity, confidentiality, and availability of AI systems. The importance of the risk
assessment in AI applications cannot be overstated. AI applications (due to their complexity and
potential for influential impacts) present unique risks that must be thoroughly assessed and
mitigated. These risks could range from data breaches to ethical concerns, such as bias in AI
decision-making.

In addition to the risk assessment, the AI system impact assessment plays a pivotal role in the
responsible use and management of AI systems. This assessment is like a business impact analysis
(BIA) for a business continuity plan. It involves assessing the ramifications, both positive and
negative, of implementing, operating, and evolving AI systems within the organization. The
assessment encompasses various dimensions to provide a comprehensive understanding of the
effects of AI systems on the organization and its stakeholders. AISIA examines the implications on
different facets of the organization, such as:

Technical Impact: Assessing the technical implications of AI systems on existing infrastructure, data
management practices, and integration with other systems.

Operational Impact: Evaluating how AI systems impact day-to-day operations, workflows, and
decision-making processes within the organization.

Ethical and Social Impact: Examining the ethical considerations and societal implications of AI
systems, including issues related to bias, fairness, privacy, transparency, and accountability.

Legal and Regulatory Impact: Identifying legal and regulatory requirements governing the use of AI
systems, ensuring compliance with laws related to data protection, intellectual property rights,
consumer rights, and industry-specific regulations.

Financial Impact: Estimating the financial implications of implementing and maintaining AI systems,
including upfront investment costs, operational expenses, and potential cost savings or revenue
generation opportunities.

Reputational Impact: Assessing how AI systems may affect the organization’s reputation and brand
image, considering factors such as trust, credibility, and public perception.

Stakeholder Impact: Analyzing the impact of AI systems on various stakeholders, including

employees, customers, partners, suppliers, regulators, and the broader community.

Evaluating the impact of AI systems is necessary to ensure that the benefits outweigh the
potential harms. Organizations can gain insights into the potential consequences of AI initiatives and
make informed decisions about deployment strategies, risk mitigation measures, and resource
allocation.

ISO 27001 and ISO 42001 provide structured frameworks for AI risk and impact assessments. These
standards offer guidelines for formulating an AI risk treatment plan and defining a process for AISIA.
They also require that the results of these assessments be documented and communicated within
the organization and made available to interested parties as appropriate. Organizations can ensure
that their AI systems are managed effectively and responsibly, with a clear understanding of the
associated risks and impacts when accomplishing an AISIA.

Processes, Competence & Skilled Workers

Skilled professionals in AI management are the backbone of any successful AI implementation. They
possess the technical expertise and strategic insight to navigate the complexities of AI systems. The
role of processes in AI systems are equally significant. Well-defined and efficient processes are
necessary for the smooth operation of AI systems, ensuring that they function optimally and deliver
the desired outcomes. These processes encompass everything, from the determination of
organizational objectives and risk management to the management of suppliers and third parties
that provide or develop AI systems.

ISO 27001 and ISO 42001 standards provide a structured framework that ensures competency and
defines processes for AI management. By integrating these standards into their AIMS, organizations
can ensure that they have the right personnel and processes in place to effectively manage AI
systems.

ISO 42001 standards includes the following key components for human oversight involved in AI
systems:

Training

Training is fundamental to ensure that all personnel involved in the AI system’s lifecycle are
equipped with the necessary knowledge and skills. This encompasses understanding the ethical
implications, technical aspects, and operational procedures related to AI systems. Training programs
should be tailored to the specific roles of individuals and updated regularly to reflect the latest
developments in AI technology and regulatory requirements. ISO 42001 emphasizes the need for
diverse expertise and competences in the development, deployment, and management of AI
systems, highlighting the importance of continuous learning and adaptation.

Roles & Responsibilities

Clearly defined roles and responsibilities are fundamental for accountability and effective
management of AI systems. ISO 42001 provides guidance on allocating roles according to the
organization’s needs, ensuring that all relevant areas are covered. By defining responsibilities,
organizations can ensure that each aspect of the AI system’s lifecycle is overseen by individuals with
the appropriate expertise and authority.

AI Champions

Designating AI champions within the organization can significantly enhance the governance and
oversight of AI systems. These individuals or teams are tasked with advocating for responsible AI
practices, facilitating communication between different departments, and leading the
implementation of AI management systems. AI champions play a pivotal role in bridging the gap
between technical and non-technical stakeholders, ensuring that AI initiatives align with the
organization’s ethical standards and strategic objectives.

Performance & Continual Improvement of AI Systems

The ISO 42001 standard provides a comprehensive framework for continuous improvement and
performance monitoring, which includes tracking general errors and failures, as well as assessing
whether the AI system is performing as expected with production data. The standard also highlights
the need for continuous evaluation and enhancement in AI systems.

Some AI systems evolve their performance as a result of machine learning (ML), where production
data and output data are used to further train the model. In such cases, continuous monitoring
confirms that the AI system continues to meet its design goals and operates as intended. The
standard recognizes that the performance of some AI systems can change, even if they do not use
continuous learning, usually due to concept or data drift in production data. This emphasizes the
importance of regular performance evaluation and the need for retraining to ensure that the AI
system continues to meet its design goals.

Overall, the standards provide a structured approach to monitor and improve AI performance. By
adhering to the standards, organizations can ensure that their AI systems maintain reliability,
efficiency, and continually improve, driving business growth and innovation.

Key components of ISO 42001 that highlight these aspects include:

Deployment and Release Criteria: Mandates organizations to establish requirements before the
release and deployment of AI systems, ensuring readiness and compliance with performance
standards.

Operation and Monitoring Controls: Requires defining elements for ongoing operation, including
monitoring, repairs, updates, and support, enabling prompt identification and resolution of issues.

Performance Criteria and Metrics: Considers operational performance and defines relevant metrics,
such as error rates and processing duration to ensure efficiency and accuracy.

Feedback of AI Systems

Organizations rely on feedback loops to help provide continual improvement to the AI systems and
to correct issues in a timely manner. They serve as a mechanism for continuous learning and
improvement, enabling AI systems to adapt and optimize their performance over time. Feedback
loops can be seen as the heart of AI systems, pumping information back into the system to refine
and enhance its operations.

The ISO 42001 standard recognizes the significance of these feedback loops, particularly in the
context of AI systems that continuously learn and change their behavior during use. The ISO
standards provide comprehensive guidelines for collecting and utilizing feedback in AI systems. This
includes processes for managing concerns related to the trustworthiness of AI systems, such as
security, safety, fairness, transparency, data quality, and the quality of AI systems throughout their
lifecycle.

Feedback collection and utilization are integral to these processes. The ISO standards guide
organizations on how to integrate feedback mechanisms into their AI management systems,
ensuring that the AI systems remain aligned with the organization’s objectives and the expectations
of various interested parties. By adhering to these ISO standards, organizations can ensure that their
AI systems are not only effective and efficient but also responsible and accountable.

A well-structured feedback loop should address several key components to be effective:

Monitoring Performance and Compliance: Continuously monitor the AI system’s performance

against technical criteria and ensure compliance with customer requirements, and applicable legal
and regulatory requirements.
Identifying and Responding to Errors and Failures: Include mechanisms for identifying errors,
failures, and performance issues and detail processes for prompt response, repairs, and corrective
actions.

Handling Continuous Learning and Data Drift: Monitor the system’s evolution through machine
learning with production data, ensuring it continues to meet design goals and address concept or
data drift.

System Updates and Modifications: Include processes for regularly updating the AI system,
addressing critical issues, compliance updates, and operational changes while minimizing
disruptions.

Support and Incident Management: Outline support processes, incident reporting, response, repair
processes, and service level agreements (SLAs) to manage and resolve issues efficiently.

Review and Improvement: Facilitate regular reviews of the AI system’s performance, effectiveness
of monitoring and response mechanisms, and compliance with intended use and regulatory
requirements.

Stakeholder Engagement: Involve relevant interested parties in monitoring and improvement

processes to consider perspectives and needs, ensuring ongoing alignment with organizational and
regulatory standards.

Corrective Actions of AI Systems

Corrective actions address deficiencies and ensure the responsible use of technologies. AI systems,
while powerful, are not infallible and can sometimes produce errors or biases that need to be
corrected. The ISO 27001 and ISO 42001 standards provide clear guidance on how to implement
corrective actions in AI management. They advocate for a proactive approach, where potential risks
are identified and mitigated before they can cause significant harm. The ISO standards also stress
the importance of continuous monitoring and regular reviews to ensure that the implemented
corrective measures are effective.

The ISO 42001 standard provides a structured approach to corrective action when
nonconformities occur. The five components of corrective action under ISO 42001 include:

Immediate Reaction: React promptly to control and correct the nonconformity, addressing its
consequences.

Evaluation: Review the nonconformity, determine its causes, and assess potential recurrence or
occurrence elsewhere.

Implementation of Actions: Implement necessary actions to address identified causes of the

nonconformity.

Review of Actions: Review the effectiveness of corrective actions taken to ensure adequate
resolution.

System Changes: Incorporate learnings and improvements from the corrective action process into
the AI management system as needed.

These components ensure that corrective actions are not only reactive but also proactive, aiming
to continually improve the AI management system.
AIMS Standard

In essence, the ISO 27001 and ISO 42001 standards serve as a roadmap for organizations navigating
the complexities of AI. They provide a clear direction for organizations to follow, ensuring that their
journey with AI is not only successful, but also responsible and ethical. It is highly recommended for
organizations to adopt these standards to promote responsible utilization and development of AI
systems. Organizations can harness the power of AI, while mitigating the risks and challenges
associated with its use, ultimately driving business growth and innovation.