Internal Audit 4.

0
Unlocking the power of digital through Artificial Intelligence
November 2023
Internal Audit 4.0
Purpose driven, digitally powered Digitalising
Internal Audit
We believe digitalisation of Internal Audit is
critical to achieving value, yet it remains a
Purpose Purpose Vision Strategy significant gap and opportunity for many
functions.

Remit High impact areas of focus CORE OPERATING SYSTEM Digitalisation refers to the integration of
enabling technologies and ways of working
Assure Advise to transform existing Internal Audit practice.
Position & organization Digital is not the goal, but it’s what it can
help you achieve be that maximising the
Core processes Risk & assurance strategy
value provided to stakeholders, enhanced
quality, greater levels of assurance,
Greatest risks External & leading practice
increased impact, greater productivity, or
Process & technology new efficiencies.
Decision governance
Anticipate For Internal Audit leaders and teams, it
Change risk Emerging & dynamic risk requires a cultural shift and a mindset of
experimentation, innovation and challenging
People & knowledge the status quo.
Behaviours
Accelerate Whilst many functions have made good
Second line functions progress in the adoption aspects of digital
Organizational learning such as analytics and visualisation, most
Digital technologies Performance & communication Internal Audit functions are yet to truly
Management action
harness the power of digital.

Agile Mindset Digital

© 2023 Deloitte LLP. All rights reserved. 2

Internal Audit’s digital landscape

Artificial Intelligence Visualisation

The simulation of human Creating insight and impact through
intelligence in machines engaging visualization of data to
programmed to think, reason, inform, support and report Internal
learn, and problem-solve Audit’s activity

Digital
Automation Analytics
Automation of routine and labour The analysis and use of data to help provide
intensive tasks to help drive assurance, advice and insight across risk
productivity, efficiencies and domains and Internal Audit data
improved quality

Cloud & platforms Process mining

The technology ecosystem, Visualising representations of business

applications and cloud based tools processes and workflows to drive insights into
such as Audit Management Systems, governance, risk and control design and
SaaS solutions, and integrations to effectiveness
enterprise systems

Digital is not the goal… it’s what it can help you achieve

Insight Quality Collaboration Efficiency Impact

© 2023 Deloitte LLP. All rights reserved. 3
 Artificial Intelligence
Understanding the terminology
Artificial Intelligence is a broad ‘umbrella’ term given to the field of computer science that focuses on creating systems that can perform tasks requiring human intelligence.

Most people are already using forms of artificial intelligence in their daily life without realising it. For example, tools like autocomplete, spellcheck, smart calendar scheduling, and suggestions on the most effective
ways to visualise data in applications such as Power BI, are all powered by forms of artificial intelligence. However, the emergence of accessible Generative AI tools, and in particular, Large Language models such as
Open AI’s ChatGPT, have excited the world around the potential these tools offer. Before jumping straight to these, it’s helpful to gain some digital fluency around AI. The below diagram helps show how terms like
machine learning and generative AI sit in the context of AI.

AI
Artificial Intelligence (AI)
The capacity for machines to perform tasks that simulates human intelligence such as data
processing, robotics and problem solving.
ML Machine Learning (ML)
ML trains models to identify and predict patterns based on human processed data, rather than
relying on hard-coded rules.

DL Deep Learning (DL)

DL is a powerful and advanced ML paradigm that leverages neural network models to improve
model performance without feature engineering.

Generative AI (Gen AI)

Gen AI is a highly sophisticated subset of AI using large parameter models to create data across a

Gen AI variety of modalities e.g. text, images, audio, video.

© 2023 Deloitte LLP. All rights reserved. 4

Artificial Intelligence
Major approaches and capabilities

Examples of how AI learns and works Examples capabilities provided by AI approaches

Machine learning Neural networks Natural Language Processing Large Language Models Speech recognition

A subfield of AI where algorithms learn A subset of machine learning The algorithmic ability of a Specialized machine learning The process of using algorithms and
from data to make predictions or inspired by the structure and machine or computer system models for used in NLP tasks, software techniques, including ML, to
decisions. The primary types of ML function of the brain, which are to process large amounts of including but not limited to NLG. recognize, focus on, and accurately
include: often designed to recognize human or “natural” language. They are designed to understand transcribe human speech into text and
patterns in data. and generate human like text. other data points.
1.Unsupervised learning Examples include data
Examples include image and extraction, language Whilst they are often used for
UL discovers patterns in data, even though Example: transposing meeting audit into
speech recognition, and text translation, sentiment text generation, their capabilities
no explicit feedback or labelled examples transcripts, personal assistants, etc.
classification. analysis, and chatbots. extend to other tasks like text
are provided as they are in supervised classification, sentiment analysis,
learning. translation and question- Perception and sensing
Deep Learning
Natural Language Generation answering. Allows machines to interpret and make
2.Supervised learning decisions based on visual data.
A specialized subset of neural
A subfield of natural language LLMs are usually based on deep
Learning from example. SL uses labelled networks that deals with
processing focused on learning architectures and are Examples include facial recognition, object
training data to learn a model or a networks containing multiple
generating coherent and trained on massive datasets. detection, and medical imaging.
function to produce the correct output (usually many) layers, known as
deep networks. DL enables contextually relevant text
given an input. based on a set of conditions,
complex ML tasks to be Robotics
completed. rules, or input data. Robotics uses a variety of AI approaches,
3.Reinforcement learning such as computer vision and machine
Examples include natural Examples include automated learning, to help robots understand their
language processing/generation, reporting, chatbots, and text surroundings and interact with them,
RL is learning by doing or by trial and
autonomous vehicles, advanced summarization. often performing complex activities.
error. It enables an agent to become
proficient in an unknown environment, image recognition, etc.
given only sensory input and occasional
rewards or punishment. © 2023 Deloitte LLP. All rights reserved. 5
Generative AI
An overview
Generative AI describes algorithms which can be used to create new content, including audio, code, images, text, simulations, and videos. Generative AI systems fall under the broad category of machine learning
and include large language models, but also many other applications. Generative AI is moving at pace and there are new fields and a huge number of organizations developing AI platforms, with both narrow and
broad capabilities.

Common types of generative AI capability include:

Image Video Voice Music Text Code

creation creation generation generation generation generation

e.g. text to image e.g. text to video, e.g. text to e.g. AI generated e.g. Content e.g. intelligent
REACHING ALL
editing speech music, lyrics, generation, copy, coding, INDUSTRIES
genres editing generation and
review

Generative AI has the potential to impact all industries with an infinite number of business applications and use cases.

Text, code, video and voice generation offer the most likely opportunities for Internal Audit functions to leverage Generative AI
in order to create impact and reveal new possibilities.

© 2023 Deloitte LLP. All rights reserved. 6

The risks of using Gen AI tools
Generative AI is impressive. The ability of technologies such as large language models to provide coherent and insightful responses, in a human like dialogue, can quickly lead to high levels of trust being placed in
their output. Functions who choose to ignore these technologies will fall behind, but their adoption of them must be managed and considered.

Accuracy Traceability Intellectual property Data security Bias Propagation

Gen AI can sometimes generate Gen AI tools are often "black Sensitive or confidential
Tools may have been trained on Gen AI can perpetuate and even
outputs that are factually boxes," making it difficult to information could inadvertently
and plagiarize existing content, amplify existing biases in the data
incorrect, misleading or understand how they arrive at leave the organisation where
leading to potential legal issues. they were trained on
inappropriate particular outputs public tools are used

Maintain a healthy Fact check the outputs Use Gen AI as an accelerator Set clear guidelines on
level of professional of Gen AI with subject rather than a replacement when and how staff
scientism matter experts of human involvement should use such tools

© 2023 Deloitte LLP. All rights reserved. 7

Generative AI for Internal Audit
How it could help…
Generative text, code and video technologies, including Large Language Models like ChatGPT, can offer a vast range of possibilities for Internal Audit. Their use is only limited by the imagination and creativity of
teams. However, we have outlined some example use cases of how Internal Audit groups can embrace these across the audit lifecycle below.

Risk assessment Audit plan development Engagement Planning Execution Reporting

Supporting auditor research and Supporting auditor research and Supporting auditor research and Analysis of data through natural Initial draft report
understanding of risk for a specific understanding on risk, business understanding on risk and business language questioning
industry process and expected controls in process in advance of planning Initial draft report review and QA
advance of engagement planning. Suggested interview questions for
Supporting audit universe creation Suggest control objectives and test different stakeholders Editorial QA e.g. simplifying
e.g. guidance on universe design, Suggested audits against the risk procedures based on in scope risk language, sentiment analysis
process universe, etc. assessed audit universe. areas Critical assessment of risk and
control descriptions (e.g. if it covers Summation of reports for Audit
Suggested scheduling and resource Suggest data sources, tables and who, what, where, when, etc.) Committee summaries
allocation based on known potential analytics tests
constraints e.g. number and levels Initial draft of workpaper
of staff Generation of video/audio
Generate scripts for data extract reporting
and analytics execution Drawing themes from interview
notes / audio Customised communications
First draft of scope/terms of stakeholders style
reference Summation/interrogation of audit
evidence documents Report language translation

Initial workpaper review and QA Drafting emails to communicate the

audit report
Initial draft of issue/observations

© 2023 Deloitte LLP. All rights reserved. 8

Leveraging the full power of digital
Combining AI with other technologies…

Artificial Intelligence By combining the power of AI with other technologies such as automation, analytics, visualisation,
process mining and platforms including the Audit Management System, third party SaaS and
enterprise systems, the potential for new capabilities that will drive greater efficiencies, quality, and
impact is huge. A small range of potential capabilities unlocked through digital are below.

Automation Cloud & platforms Intelligent Intelligent AI supported

Personalised
Predictive text to interview analysis analytics; Outlier detection control design
help drive exception
to identify risk, suggested and with machine through digital
consistency and workflow
themes and self-service learning twins and
quality management
insight analytics synthetic data

Optimising risk
Resource and Automated Continuous and Automated Text mining &
coverage through
schedule workpaper automated follow up process sentiment
intelligent plan
optimisation creation and QA control testing and reporting analysis
development
Data Analytics Process mining

Professional
Customised L&D
Automated skills practices chatbots
pathways and Personalised co- AI supported Interactive and
and capabilities to advise on
just in time pilot assistants quality review visual reporting
management methodology and
Visualisation training
QA topics

© 2023 Deloitte LLP. All rights reserved. 9

Approaching Generative AI solutions
Use, shape or build?
Data entered into open consumer versions of generative AI tools is used to train the models that drive them (the usual trade-off for free services). Organisations want to take
advantage of Generative AI solutions but on their own terms. Should they use open tools, shape capabilities developed by others or build their own?

Accessing open AI solutions Shaping AI solutions Building from scratch

Generative AI can be easily consumed through either an API, Organisations can leverage existing foundational models and Building AI solutions from scratch is technically difficult and
like ChatGPT, or through another application, like GitHub Co- retrain them (a process known as fine-tuning) with your own costly for companies that haven’t yet made significant
pilot (for software acceleration). data. This can reduce ‘hallucinations’ and provide more investment in generative AI. It requires significant expertise,
accurate and relevant results. Fine-tuning applies to both advanced infrastructure, and few will have access to the levels
Given the pace of technology development, buying into hosted cloud LLMs and open source LLM models you run of petabyte-scale datasets needed to initially train these
consumer solutions ensures new features and plug-ins are yourself, so this level of ‘shaping’ doesn’t commit you to one models to levels of capability seen by open solutions.
readily accessible as they are developed. approach.
By the time it would take an organisation to design and build
Many tools are being integrated in applications which staff To benefit from fine-tuning, organisations need to create a their own, efforts could be better applied to shaping existing
already know-how to use e.g. Microsoft Co-pilot in Office 365 data environment that can be consumed by the model. This models or making use of general capability in open solutions,
can require significant data, good quality data, sufficient and exploring how your organisations processes could evolve
Consumer tools can offer significant benefits from their infrastructure and data engineering and machine learning using these new capabilities.
generalised capability but little competitive differentiation due capabilities.
to the lack of customisation.
Hence AI solutions that you calibrate within your four walls will
likely to be much smaller in size and narrow in application. For
those who do invest in shaping generative AI systems created
from existing models and services, these tools most likely to
offer competitive differentiation.

© 2023 Deloitte LLP. All rights reserved. 10

What do to now
Immediate steps Internal Audit leaders should take

Increase your digital fluency

01 Starting engaging with learning and development now. You don’t need staff to become data scientists, engineers or digital experts. However, being familiar with
the terminology, types of capability and potential for these tools will help accelerate adoption and get your function thinking about the use cases.

Engage with your technology teams

02 Understand your organization’s stance toward artificial intelligence, both from a data privacy and security perspective for open solutions, and it’s appetite for
shaping existing solutions within the safety of your organization’s environment.

Determine your digital strategy and potential

03 Determine how digital can help you achieve your broader functional strategy and outcomes. Systematically review your ways of working to identify potential use
cases, explore return on investment and pilot through experimentation.

Clean up your (and the organisation’s) data

04 The quality of AI both in terms of its training and it output will be a product of the quality of data it is given and looking at. Many organizations (including Internal
Audit) have poor data quality, version control or out of date versions of documents that haven’t been removed from intranets for years. Whilst you’re waiting for
some of the tools to become more accessible, getting your house in order will pay dividends to the value AI can deliver.

© 2023 Deloitte LLP. All rights reserved. 11

How we can help
Deloitte’s Digital IA services

© 2023 Deloitte LLP. All rights reserved. 12

Digital is not the end goal. It’s what it can help you achieve

Why Digital matters? Where to start

Challenges facing Internal Audit today
Facing ever growing demands and expectations, Internal Audit Be purpose driven
leaders must evolve to keep pace. We believe digitalisation of
Internal Audit is critical this. Digitalisation is a journey that has the potential to transform
Internal Audit’s ways of working, but it has greatest impact when
Whilst some functions have made good progress, most are yet to Sourcing and retaining talented analytics resources its outcomes and benefits are aligned with the function’s
truly harness the power of digital. It remains a significant gap with the right mix of IA knowledge, business acumen purpose. Internal Audit leaders must have clarity on their
and opportunity. and technical expertise broader purpose, vision and the desired outcomes before
embracing digital.
Digitalisation refers to the integration of enabling technologies
to transform existing Internal Audit practices. But digital is not Digital fluency is important, but mindset drives true value
the goal. It’s what it can help you achieve that is key; maximising
the function’s value for, and impact on, the organisation. Digital Upskilling is critical. Auditors need to develop the necessary skills
Desire to incorporate leading edge AI and Machine
is a path to enhancing quality, providing greater levels of to recognise opportunities for analytics, automation, AI etc. but
learning but limited ideas, skills, and knowledge hold
assurance, revealing insights, achieving new levels of embedding a culture of innovation and continuous
the IA function back
productivity and a tool to reimagine the art of the possible. improvement, where challenging the status quo is expected, is
key to revealing new possibilities.
The digital landscape is broad, covering artificial intelligence,
automation, audit management systems, cloud-based solutions, Systematically assess your ways of working
visualisation, data analytics, and tools such as process mining.
Whilst they can be deployed in isolation, the power of digital is in Delivering on innovation programs fail where a mix of The change will take time, but you can’t achieve anything
their combination. low volume ideas and overall poor governance leads to without taking your first step on the journey. With clarity of
program collapse purpose and desired outcomes as your guide, systemically assess
Digitalisation requires a holistic and coordinated approach across your ways of working to identify, evaluate, pilot and deploy
the function, a clear vision, and the means to achieve it. And digital opportunities across the lifecycle.
therein lies the challenge…
Data is still the key
Digitising requires a level of technology fluency, deep knowledge
of Internal Audit practice, and a curiosity to explore the ‘art of IA faces a need to upskill talent across the function on With the right data, digital technologies can drive new levels of
the possible’. To embrace digital, Internal Audit leaders need to emerging technologies, data science, and project insight, assurance, efficiencies and experience for your
have and establish a mindset of experimentation, innovation and management stakeholders. Alignment with the enterprise’s technology and
challenging the status quo across the function. It’s a cultural shift data strategy can help accelerate Internal Audit’s progress.
as much as a technology lift. © 2023 Deloitte LLP. All rights reserved. 13
Our approach to digitising Internal Audit

Setting your digital strategy Developing your digital skills and capabilities

Through our experienced facilitators, our digital strategy labs help you To help you grow the skills needed for your digital ambition, we can support you in development of your digital
connect your digital ambition with your function’s purpose, role and competency framework and skills assessment. Through our Internal Audit Development Pathways, we can give you access
remit. We understand your desired outcomes, and help you identify the into tried and tested learning. Equally, we can design, build and deliver bespoke digital pathways to provide learning and
strategic priorities needed to unlock the power of digital. We development activities for your teams, specialists and leaders.
benchmark your digital capabilities today using our Digital IA maturity
framework.

We show you the art of the possible, providing you insights on leading
and innovative practice, and share our experience in how to transform
your function successfully.
Helping you identify, design and build digital use cases

Opportunity identification - Based on a clear understanding of your desired outcomes, we systematically review and
Step by step, we guide you through our Digital IA framework, designed
assess your function to identify opportunities to leverage enabling technologies inc. automation, AI, analytics,
to help you consider and define your roadmap across multiple areas
visualisations, common AMS functionality, etc.
including:
Intake & pipeline management – We help you engage your team to capture ideas quickly and frequently, consider
Strategy – your digital vision, value drivers, organisational alignment,
return on investment and prioritise your digital journey.
partnerships and eminence.
Development and deployment – we can give you access to our dedicated technology professionals who provide scale to
People - the skills, culture, team structures and behavioural attitudes
your internal audit team, without the long-term commitment of full-time hires, to accelerate your development and
needed to build your digital capabilities.
deployment of digital assets.
Process - the processes required to establish and industrialise digital
Maintenance, recertification, decommissioning and reporting – digital assets need ongoing management. We help you
ways of working across your function to realise strategic change.
design and stand up the governance, processes and monitoring activities needed to support your digital ways of
working.
Technology - the relevant tools, technology and infrastructure required
to support a digitally enabled function.

Data - the sources, access and potential uses of data across the Internal
Audit lifecycle to drive tangible value.

© 2023 Deloitte LLP. All rights reserved. 14

Connect with us to start your digital journey

Current trends and focus areas Investing in internal technology and digital opportunities can result in:
Whilst there is no one-size-fits-all approach to digitalisation, common opportunities have emerged across the Internal
Audit lifecycle. For example: Increased automation:
01 Reduction of manual processes and increased assurance coverage
➢ Audit universe visualisation ➢ Audit procedure automation providing analysis and
workpaper population for in scope risks and controls Significant cost savings:
➢ Data driven, automated risk assessment
02
Efficiency gains and cost savings throughout the audit lifecycle
➢ AI powered search engines and knowledge management
Enhanced analytics:
➢ Self-service analytics
03 Advanced analytics capabilities across each audit phase with centralized data
➢ Large Language Model powered document interrogation, source and greater insight
➢ Machine Learning for anomaly detection, clustering & and content creation
predictive analytics 04 Staff turnover reduction:
➢ Automated follow up process and reporting Significant reduction in IA staff turnover and improved skillsets
➢ Automated data extraction and controls testing
➢ Automated Audit Committee paper production 05 Quality customer experience:
Strengthened IA stakeholder relationships across the organization
Contact us to learn more about unlocking the power of digital.

Meet our digital and innovation leadership:

Heinz Wustmann Frank Wehrle Jan Grüne Andreas Böhner

Offering Lead Internal Audit Operational IA and Supply Chain Digital IA and Analytics Core IA and Co-Sourcing
[email protected] [email protected] [email protected] [email protected]

This publication has been written in general terms and we recommend that you obtain professional advice before acting or refraining from action on any of the contents of this publication.
Deloitte LLP accepts no liability for any loss occasioned to any person acting or refraining from action as a result of any material in this publication.

Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 1 New Street Square, London, EC4A 3HQ, United
Kingdom. Deloitte LLP is the United Kingdom affiliate of Deloitte NSE LLP, a member firm of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee (“DTTL”). DTTL and
each of its member firms are legally separate and independent entities. DTTL and Deloitte NSE LLP do not provide services to clients. Please see www.deloitte.com/about to learn more about
our global network of member firms.

© 2023 Deloitte LLP. All rights reserved. 15

This publication has been written in general terms and we recommend that you obtain professional advice before acting or refraining from action on any of the
contents of this publication. Deloitte LLP accepts no liability for any loss occasioned to any person acting or refraining from action as a result of any material in this
publication.

Deloitte LLP is a limited liability partnership registered in England and Wales with registered number OC303675 and its registered office at 1 New Street Square,
London, EC4A 3HQ, United Kingdom.

Deloitte LLP is the United Kingdom affiliate of Deloitte NSE LLP, a member firm of Deloitte Touche Tohmatsu Limited, a UK private company limited by guarantee
(“DTTL”). DTTL and each of its member firms are legally separate and independent entities. DTTL and Deloitte NSE LLP do not provide services to clients. Please see
www.deloitte.com/about to learn more about our global network of member firms.

© 2023 Deloitte LLP. All rights reserved.