Performance Audit: ISSAI Implementation Handbook

Performance
Audit
ISSAI Implementation
Handbook
Version 1, August 2021
TABLE OF CONTENTS
Quality Statement for Performance Audit ISSAI Implementation Handbook Version 1 (8 June 2021) 7
About the handbook 9
Background 9
What is the purpose of the handbook? 9
How was the handbook developed? 10
Contents of the handbook 10
Chapter 1 12
What is performance auditing? 12
What is performance auditing? 12
What are the objectives of a performance audit? 13
What is the relationship among the 3Es? 17
What value do performance audits bring? 19
What types of reports result from performance audits? 21
What is the difference between performance audit and other types of public audits? 23
Who are the three parties in a performance audit? 26
What is subject matter and subject matter information? 27
Chapter 2 28
What are the key principles of quality performance audits? 28
What is quality control? 29
What are independence and ethics? 31
What are professional judgement and scepticism? 38
What is audit team competence? 40
What is materiality? 43
What are audit documentation and audit supervision? 44
What are audit risk and assurance? 49
What does communication with audited entities, external stakeholders, media and the public involve? 52
IDI’s considerations to mainstream inclusiveness and maximise the impact of performance audits 57
Chapter 3 64
How do you select performance audit topics? 64
What is the strategic planning process? 65
How do SAIs scan the audit environment to identify possible topics for performance audits? 67
How do external stakeholder requests arise? 71
Why might a SAI consider auditing a topic that is not the responsibility of a single audited entity? 72
What criteria do SAIs use to select topics for performance audits? 73
Chapter 4 79
How do you design a performance audit? 79
How do you conduct a pre-study of the audit topic? 80
How do you determine the approach for a performance audit? 84
How do you develop the audit objective(s)? 86
How do you formulate audit questions? 89
How do you determine the scope of the audit? 92
How do you select audit criteria? 94
How do you develop the audit methodology? 100
How do you manage risk during audit design? 104
How do you determine the time frames and resources needed for a performance audit? 107
How do you document the audit plan? 108
How do you involve internal and external stakeholders and management when designing a performance audit?
112
Chapter 5 116
How do you conduct a performance audit? 116
How do you determine the sufficiency and appropriateness of evidence? 117
How do you gather information for a performance audit? 122
How do you analyse information? 146
How do you document and safeguard information? 156
Chapter 6 159
How do you develop findings, conclusions and recommendations? 159
What is an audit finding? 160
How do you compare audit criteria to condition? 162
How do you determine cause and effect? 163
How do you assess your evidence? 165
How do you develop conclusions and recommendations? 169
How do you prepare for drafting your report? 175
Chapter 7 177
How do you write a performance audit report? 177
How do you develop a performance audit report? 177
What are the main attributes of a performance audit report? 178
How do you create a logical report structure? 182
How do you ensure the quality of the report? 194
How do you consider audited entities or third party comments? 197
How do you publish the final report and communicate the results? 198
Chapter 8 200
How do you follow-up on audit results? 200
What is performance audit follow-up? 200
How do you conduct follow-up? 202
When do you conduct follow-up? 204
How do you determine the impact of the audit? 206
How do you report the results of follow-up? 207
Appendices 211
Appendix 1: Example of an SAI QA framework for assuring compliance with applicable standards 212
Appendix 2: Example of a permission to engage in outside activities form 213
Appendix 3: Example of an independence statement 214
Appendix 4: Example of an audit topic selection matrix 215
Appendix 5: Sample design documents 216
Appendix 6: Design paper checklist 226
Appendix 7: Project schedules and work breakdown structure 227
Appendix 8: Interview guide 231
Appendix 9: Example of a record of analysis 236
Appendix 10: Example of a data reliability assessment 238
Appendix 11: Sample data reliability questions for the audited entities 241
Appendix 12: Sample data collection instrument 243
Appendix 13: Sample template for documenting direct observations 244
Appendix 14: How to conduct a survey 245
Appendix 15: Content analysis 248
Appendix 16: Sample template for documenting a summary 250
Appendix 17: Example of a regression analysis 252
Appendix 18: Sample GAO highlights page 254
Appendix 19: Sample European Court of Auditors executive summary 255
Appendix 20: Description of an audit methodology in a performance audit report 256
Appendix 21: Illustration of an action plan and a follow-up desk review template 257
List of figures
Figure 1: Relationship between the 3Es .......................................................................................................................... 18

Figure 2: How performance audits can add value ........................................................................................................... 20
Figure 3: Themes that appear in performance audits...................................................................................................... 22
Figure 4: Key elements of quality control for performance auditing ............................................................................... 30
Figure 5: Six major threats to independence during a performance audit ....................................................................... 32
Figure 6: Common risks in performance auditing ............................................................................................................ 50
Figure 7: Value chain of SAI outputs and SAI outcomes .................................................................................................. 57
Figure 8: How SAIs could select audit topics ................................................................................................................... 66
Figure 9: Methods of identifying possible audit topics .................................................................................................... 68
Figure 10: How SAI Zambia scans the environment to identify potential audit topics ..................................................... 71
Figure 11: Illustrative list of selection criteria for audit topics ......................................................................................... 75
Figure 12: Sample summary of an audit topic ................................................................................................................. 77
Figure 13: Information sources ....................................................................................................................................... 82
Figure 14: Relations when auditing more than one entity ............................................................................................... 86
Figure 15: Examples of appropriately and inappropriately-formulated audit objectives by audit approach .................... 88
Figure 16: Types of evaluative audit questions by audit approach .................................................................................. 90
Figure 17: Example of issue analysis ............................................................................................................................... 91
Figure 18: Example of cause – effect problem analysis ................................................................................................... 92
Figure 19: Scope questions ............................................................................................................................................. 93
Figure 20: Attributes of suitable criteria ......................................................................................................................... 97
Figure 21: Criteria and corresponding audit questions .................................................................................................... 98
Figure 22: Benefits and considerations of select information collection methods ......................................................... 101
Figure 23: Benefits and considerations of select data analysis methods ....................................................................... 103
Figure 24: Design matrix template ................................................................................................................................ 110
Figure 25: Sufficiency of evidence................................................................................................................................. 118
Figure 26: Appropriateness of evidence........................................................................................................................ 118
Figure 27: Types of evidence ........................................................................................................................................ 126
Figure 28: Examples of direct observations or physical inspections .............................................................................. 139
Figure 29: Examples of common types of qualitative analysis ....................................................................................... 147
Figure 30: Basic concepts in descriptive statistics ......................................................................................................... 151
Figure 31: Incomes of staff at a factory ......................................................................................................................... 152
Figure 32: Road accidents in regions with different types of inspections ...................................................................... 154
Figure 33: Examples of charts that can be used for data visualization ........................................................................... 156
Figure 34: Elements of a finding ................................................................................................................................... 160
Figure 35: Mind map .................................................................................................................................................... 166
Figure 36: Fishbone chart ............................................................................................................................................. 167
Figure 37: Organisational chart ..................................................................................................................................... 168
Figure 38: Audit findings matrix template ..................................................................................................................... 173
Figure 39: Illustration of one finding of an audit findings matrix ................................................................................... 174
Figure 40: Examples of visuals displays of information in performance audit reports ................................................... 181
Figure 41: Contents page using question-based headings ............................................................................................. 184
Figure 42: Contents page using descriptive headings .................................................................................................... 185
Figure 43: Using a simple graphic to illustrate the responsibilities of audited organisations ......................................... 188
Figure 44: Illustrations of audit conclusions .................................................................................................................. 192
Figure 45: Examples of recommendations in select SAI performance audit reports ...................................................... 193
Figure 46: Example of a data reconciliation document for a performance audit report ................................................ 195
Figure 47: How NAO Tanzania carries out quality control reviews of its audit reports .................................................. 196
Figure 48: Adapted portion of a follow-up report ......................................................................................................... 209
Acronyms and abbreviations
3i Audit Programme ISSAI Implementation Initiative Programme

AFROSAI-E African Organisation of English-speaking Supreme Audit Institutions
GAO Government Accountability Office
GPG Global Public Good
GUID INTOSAI Guidance
IDI INTOSAI Development Initiative
IFPP INTOSAI Framework of Professional Pronouncements
INTOSAI International Organization of Supreme Audit Institutions
INTOSAI-P INTOSAI Principles
ISAM IDI´s SDGs Audit Model
ISSAI International Standards of Supreme Audit Institutions
LNOB Leave No One Behind
NAO National Audit Office
PAS Performance Audit Subcommittee
SAI Supreme Audit Institution
SAO State Accounting Office
SAO State Audit Office
SDG Sustainable Development Goal
TCU Tribunal de Contas da União (Federal Court of Accounts)
UNDP United Nations Development Programme
WoG Whole of Government
Quality Statement for Performance Audit ISSAI Implementation Handbook
Version 1 (8 June 2021)
INTOSAI Goal Chairs and IDI’s joint paper on ‘Quality assuring INTOSAI public goods that are
developed and published outside due process’ identifies three levels of quality assurance, as follows:
QUALITY ASSURING INTOSAI PUBLIC GOODS THAT ARE DEVELOPED AND PUBLISHED OUTSIDE
DUE PROCESS – Levels of Quality Assurance
Level 1: Products that have been subjected to quality assurance processes equivalent to INTOSAI
due process, including an extended period of transparent public exposure (90 days)
Level 2: Products that have been subjected to more limited quality assurance processes involving
stakeholders from outside the body or working group responsible for the products’ initial
development. Quality assurance processes might, for example, include piloting, testing and inviting
comments from key stakeholders, although not go as far as full 90-day public exposure
Level 3: Products that have been subjected to rigorous quality control measures within the body or
working group responsible for their development
Different levels of Quality Assurance may be appropriate for different Global Public Goods (GPG). This
GPG has been developed according to quality assurance level 1.
Quality Assurance Protocol: Version 2.01

IDI’s Protocol for Quality Assurance (QA) of IDI’s Global Public Goods defines measures to ensure
quality based on the three levels of quality assurance above. For quality assurance level 1, these
measures include: approval by the IDI Board to create the GPG; formation of a competent product
development team; peer review by experts external to the development team; modification based
on review; proofreading, editing and translation of the document by competent persons; public
exposure for a period of 90 days/consultation with relevant stakeholders representing views from
most regions, most models of auditing, developed and developing countries, and from the
perspective of global bodies; modifications of the document based on comments received during
public exposure; and due approvals for the GPG version 1.
Updates to this GPG

To ensure that this GPG stays relevant, IDI will undertake major revision of this Performance Audit
ISSAI Implementation Handbook whenever there are changes in performance audit ISSAIs. Major
revisions will follow IDI’s Protocol for Quality Assurance. In addition, light touch reviews may be
undertaken as per need. Such light touch reviews will not normally be subject to this Protocol.
1 The protocol is available at http://www.idi.no/en/idi-library/global-public-goods

This GPG is owned by IDI’s Professional SAIs work stream, which is responsible for maintenance of
this GPG.
Quality Assurance Review Process

Shourjo Chatterjee (Strategic Support Unit, IDI) has undertaken a QA review of the process followed
for the development of this GPG, against QA Protocol Version 2.0. The QA reviewer is familiar with
IDI’s protocol for QA of GPGs and was not involved in development of the GPG. This QA review process
is designed to provide all stakeholders with assurance that the IDI has carried out the quality control
measures stated above, designed to meet quality assurance level 1.
Results of the Quality Assurance Review

The QA review of the process followed in developing this GPG concluded that the Protocol has been
followed as required for quality assurance level 1 in all respects.
Conclusion
Based on the QA review, IDI assures the users of this GPG that this document has been subjected to
a quality assurance process equivalent to Due Process for INTOSAI Framework of Professional
Pronouncements (IFPP), including an extended period of transparent public exposure.
Mr. Einar Gørrissen

Director General
INTOSAI Development Initiative
8 June 2021
About the handbook
Background
In its ISSAI2 Implementation Initiative (3i programme3), in 2014, the INTOSAI Development
Initiative (IDI) developed the first Performance Audit ISSAI Implementation Handbook. It was
based on the standards in place at that time. These standards then provided the basis of the audit
methodology described in the first performance audit handbook.
The INTOSAI framework has also changed since the first IDI Performance Audit ISSAI
Implementation Handbook was published. The INTOSAI Framework of Professional
Pronouncements (IFPP) is currently organised into principles, standards, and guidance. More
information can be found at www.issai.org.
In 2016, INTOSAI approved the new Performance Audit Standard – ISSAI 3000. As the
authoritative standard for performance audit, the new ISSAI 3000 contains major changes from
earlier ones. It is organised in requirements and explanations and has a different structure,
comparing to the previous version. The INTOSAI Performance Audit Subcommittee (PAS) has also
developed guidance documents for performance audits: GUID 3910 (Central Concepts for
Performance Auditing) and GUID 3920 (The Performance Auditing Process).
Subsequently, in its support for implementing the PA ISSAI in Supreme Audit Institutions, the IDI
initiated the revision of the PA ISSAI Implementation Handbook to reflect the changes in the
ISSAI. This handbook is the result of that process, as it incorporates the current ISSAI 3000-based
audit methodology that is applicable to performance audits carried out by Supreme Audit
Institutions (SAIs). Moreover, the IDI also received feedback that SAIs needed a handbook that
the performance auditor could use in conducting ISSAI compliant audits.
What is the purpose of the handbook?
The handbook contains explanations of the ISSAI performance audit process and audit working-
paper templates that are designed to facilitate the application of ISSAIs in practice. It is not a
manual or a prescriptive performance audit methodology. The handbook intends to provide
guidance and to present one of the possible ways in which you can implement Performance Audit
2 International Standards of Supreme Audit Institutions.

3 The International Organisation of Supreme Audit Institutions’ (INTOSAI’s) Strategic Plan and the ISSAI Rollout Model (approved
by INTOSAI’s Governing Board in October 2011) mandated IDI to ‘support ISSAI Implementation’. In keeping with this mandate,
IDI launched a comprehensive capacity development audit programme called the ISSAI Implementation Initiative – 3i.
9
ISSAIs. SAIs may need to design and develop additional guidance and working-paper templates
to meet any other requirements imposed by their laws, regulations and practices.
The handbook may be used and adapted by all public sector performance auditors. The
methodology suggested in the handbook extensively covers the audit work to be performed and
documented at an audit engagement level.
The handbook may also be used by organisations supporting SAIs in developing audit
methodology for performance audit.
How was the handbook developed?
This handbook was developed as per the requirements of IDI’s Protocol for Quality Assurance of
its Global Public Goods V2.0.
It has been written by an IDI/PAS team which included PAS members from the U.S. Government
Accountability Office (GAO) and United Kingdom National Audit Office (UK NAO), PAS Chair from
Office of the Auditor General of Norway and INTOSAI Development Initiative team. More than
40 SAIs and key stakeholders have reviewed and provided feedback during the public exposure
or provided examples to illustrate some of the concepts in the handbook.
While the PAS was heavily involved in this version 1 published in June 2021, any subsequent
reviews or changes to this version will not be subject to the PAS review. The maintenance of this
handbook will be the sole responsibility of IDI, who will place mechanisms for regular review and
updates to the handbook in consultation with key stakeholders.
The IDI acknowledges and appreciates the strong partnership and valuable contribution of the
PAS Chair from the Office of the Auditor General of Norway and PAS members from the U.S. GAO
and UK NAO in the development of this handbook.
Contents of the handbook
This handbook is developed from the auditor’s perspective. It is meant for auditors who want to
use ISSAIs when conducting performance audits. It includes ISSAI-based audit methodology
intended to address the requirements of ISSAI 3000 – the international standard for performance
auditing – which applies to performance audits conducted by Supreme Audit Institutions (SAIs)
in a public-sector environment. It also includes information related to SAIs, to the extent, it is
relevant for the auditor work.
The handbook promotes global best practice. It is not an interpretation guide to performance
audit ISSAIs. It provides practical advice on how SAI auditors can comply with ISSAI requirements
10
and add value through high-quality performance audits. In using this handbook, SAIs will need to
adapt the methodology described to suit their local needs.
There are eight chapters in this handbook, covering:
• what performance auditing is;
• key principles of a quality performance audit;
• how to select a performance audit topic;
• how to design a performance audit;
• how to conduct a performance audit;
• how to develop findings, conclusions, and recommendations;
• how to write a performance audit report; and
• how to follow-up on a performance audit.
As audits are iterative processes, care was taken to maintain the linkage between different audit
stages when writing each chapter and developing associated working-paper templates, guidance
and examples. Cross-referencing related templates also ensures that users understand the need
to maintain such linkages in an actual performance audit.
The readers of this handbook may also like to refer to other IDI global products, which
complement this handbook. These include the SAI’s Performance Measurement Framework
20164, the ISSAI implementation needs assessment tool (iCAT)5 and IDI´s SDGs Audit Model
(ISAM) 20206.
4 https://www.idi.no/elibrary/well-governed-sais/sai-pmf
5 https://www.idi.no/work-streams/professional-sais/gpgs/issai-icats
6 https://www.idi.no/work-streams/relevant-sais/auditing-sdgs
11
Chapter 1
What is performance auditing?
Before beginning a performance audit, it is important to understand what a performance audit

is and how it differs from other types of audits, such as financial and compliance audits. This
chapter presents the definition and purpose of performance audits in the public sector and
identifies the value that such audits can add. The chapter also provides definitions and examples
of the dimensions we use to assess performance – economy, efficiency and effectiveness,
collectively known as ‘the 3Es’.
This chapter will answer the following questions:

• What is performance auditing?
• What are the objectives of a performance audit?
• What is the relationship among the 3Es?
• What value do performance audits bring?
• What types of reports result from performance audits?
• What is the difference between performance audit and other types of public audits?
• Who are the three parties in a performance audit?
• What is subject matter and subject matter information?
What is performance auditing?
Performance auditing
Performance auditing carried out by SAIs is an independent, objective, and reliable examination of
whether government undertakings, systems, operations, programmes, activities, or organizations are
operating in accordance with the principles of economy, efficiency and effectiveness and whether
there is room for improvement.
Source: ISSAI 3000/17
A performance audit is one of three main types of public-sector audits defined in the
International Standards of Supreme Audit Institutions (ISSAI) 100/22. It is distinct from the other
two main types, financial audits and compliance audits, as discussed later in this chapter.
Performance audits typically test if a government is making good use of resources to effectively
deliver its policy goals and achieve its intended impact. Such audits often intend to examine the
implementation of a policy or policies. SAIs may use tests to examine government performance
against suitable criteria, then try to find the reasons for any under-performance. Conversely,
performance audits may also identify what is working well within audited entities or measure
12
how performance has improved due to certain changes the entities have made to policy or
operations.
A performance audit covers the full range of government activities, including organisational,
financial and administrative systems (INTOSAI-P-1, Section 4). A performance audit may focus on
a single programme, policy, entity or fund, or may focus on outcomes or systems, looking across
programmes, policies and entities that contribute to the outcome or system. It can focus on:
• Activities, for example, procurement policies across government.

• Outputs, for example, productivity levels in government-owned industries.
• Outcomes, for example, reductions in carbon footprint due to energy efficiency policies in
government buildings.
• Delivery of services, for example, speed and quality of particular government service.
What are the objectives of a performance audit?

The main objective of performance auditing is to constructively promote economical, effective
and efficient governance and to contribute to accountability and transparency. Performance
auditing seeks to provide new information, analysis, or insights and, where appropriate,
recommendations for improvement (ISSAI 300/12, ISSAI 3000/18). By providing new analytical
insights, making information more accessible to stakeholders, providing an independent and
authoritative view or conclusion based on audit evidence, and providing recommendations based
on an analysis of audit findings, performance audits deliver new information, knowledge and
value (ISSAI 300/10).
What are the 3Es – economy, efficiency and effectiveness?

Economy, efficiency and effectiveness are central to performance auditing. They are also a good
way of distinguishing a performance audit from a compliance audit. These principles are defined
in ISSAI 300/11, and GUID 3910/35-48 elaborates on their meaning. The requirement, according
to ISSAI 3000/35, states that “the auditor shall set a clearly-defined audit objective(s) that relates
to the principles of economy, efficiency and/or effectiveness.”
13
Economy: Keeping the cost low
The Standard
Economy is minimising costs of resources used in performing an activity. The resources used should be
available in due time, in and of appropriate quantity and quality and at the best price.
Auditing economy focuses the audit on how the audited entities succeeded in minimising the
cost of resources (input), taking into account the appropriate quality of these resources. This
part of the audit focuses only on the input by asking: “Are the resources used available in due
time, of appropriate quantity and quality, and at the best price?” (GUID 3910/38).
When conducting audits of economy, the auditor may provide answers to such questions as 7:
• Have the best prices been obtained for consultancy services?
• Is there potential for reducing the cost of sickness absences?
• Are there procedures in place to ensure that transport costs of food aid are the lowest
available?
• Has there been a waste of resources in achieving an output?
Considerations of economy often lead to examining processes and management decisions
regarding the procurement of goods, works and services.
Efficiency: Making the most of available resources
The Standard
Efficiency is getting the most from available resources. It is concerned with the relationship between
resources employed (the inputs) and outputs delivered in terms of quantity, quality and timing.
Efficiency assesses the relationship between inputs and outputs. Auditing efficiency means
asking whether the inputs have been put to optimal or satisfactory use or whether the same
or similar outputs (in terms of quantity, quality and turnaround time) could have been
achieved with fewer resources. In other words, “Are we getting the most output – in terms of
quantity and quality – from our inputs?” (GUID 3910/39). Therefore, efficiency is about the
maximum output obtained for a given level of input or the minimum level of input required
7 Adapted from ECA Performance Audit Manual, 2017.
14
for a given output level. Quality is an important concept on the input side, both in efficiency
and economy (GUID 3910/38).
Efficiency is a relative concept, meaning that a process, instrument or programme is either
more or less efficient than another. For an audit on efficiency, you, need to conduct some
comparison. You may, for example, compare similar activities in comparable entities; one
process (in one entity) with the same process at an earlier point in time; a process before and
after the adoption of a policy or procedure; the efficiency of an organisation with an accepted
set of characteristics of efficient organisations. Audits of efficiency can also examine the
processes leading from input to output to expose shortcomings in these processes or their
implementation. This can lead to a better understanding of why processes are efficient, even
without measuring efficiency itself. (GUID 3910/41)
In audits of efficiency, you might ask questions such as 8:
• How does the cost per job created by a training programme for the unemployed compare
with similar costs per job elsewhere?
• Could project X have been implemented differently that would have resulted in improved
timeliness and quality?
• Are adequate procedures and criteria for prioritising and selecting transport infrastructure
projects to ensure maximum impact in place?
• Are schools maximising the use of their information technology equipment?
When the audit objective of efficiency considers outputs, you will usually focus on processes
by which an organisation transforms inputs into outputs.
Effectiveness: Achieving the stipulated aims or objectives
The Standard
Effectiveness is meeting the objectives set and achieving the intended results.
Effectiveness deals with outputs, results or impacts. It is about the extent to which policy
objectives have been met in terms of the generated output. It is concerned with the
relationship between goals or objectives on the one hand and outcome on the other. The
question of effectiveness consists of two parts: first, to what extent the objectives are met
15
and second, if this can be attributed to the output of the policy pursued (GUID 3910/42). It
focuses on questions such as:9
• Have infrastructure projects contributed to increased traffic flow while improving safety and
reducing journey times?
• Have suitable measures to monitor and mitigate the environmental impact in sector X been
set up and properly implemented?
• Are departments or entities achieving their objectives for all sectors of the community?
Audit of effectiveness will concentrate on outputs, results or impacts. When assessing
effectiveness, SAIs consider whether and how a government policy, programme or activity is
meeting its goals. Sometimes SAIs may split effectiveness into two distinct aspects:
• The attainment of specific objectives in terms of outputs (this is called efficacy in some
SAIs).
• The achievement of intended results in terms of outcomes.
For example, you may be auditing a Ministry of Education programme designed to improve
the skills of students who have left school to fill anticipated skills gaps in the workforce. If you
focus purely on outputs, your focus will probably be on the changes in indicators, such as the
number and proportion of students leaving school with the target qualifications. A more
ambitious audit, looking at outcomes, might consider more complex questions such as:
• Has the policy led to any change in the skills gap in the labour market?
• How well is the Ministry able to predict and respond to gaps in the labour market?
In that case, when you look at effectiveness in terms of outcomes, it would be necessary to
look at connections among entities and institutions. You need to consider a larger
environment. The expected outcome will not depend just on one programme or initiative. In
the example above, it might involve entities related to employment, transport, finance,
besides the entity directly responsible for the programme.
SAIs working on effectiveness can benefit from approaches drawn from disciplines such as
programme evaluation – the activity of examining the implementation and impacts of policy
interventions to identify and assess their intended and unintended effects and costs. Where
appropriate, SAIs and audit teams have to consider the impact of the regulatory or
institutional framework on the performance of the audited entities.
16
Auditing the effectiveness of performance in relation to the achievement of the audited
entities’ objectives entails auditing the actual impact of activities compared with the intended
impacts.
Effectiveness can be measured by various methods. The most sophisticated methods compare
the situation being addressed before and after the introduction of the policy or programme
and involve measuring the behaviour of a control group, which has not been subject to the
policy or programme (the counterfactual) through a randomized trial or as a quasi-
experiment.10 However, this type of method is not always feasible. Sometimes more
qualitative methods are better suited to gain insight into causal relations between policy or
programme and effect. When concluding the causal relation between policy or programme
and effects, it is important to clearly communicate the strengths and limitations of the
methods used. There are various documents providing guidance in choosing the right methods
(GUID 3910/45).
In practice, it will be difficult for you to make these comparisons, partly because suitable
comparative material is often lacking, and it can be extremely difficult to isolate the impacts
of the policy or programme being audited from other outside factors. More commonly, you
could assess the plausibility of the assumptions on which the policy is based. This is sometimes
called testing the programme theory. You could also assess if earlier steps in the programme
– especially steps necessary for the final impact – have been achieved. Often, a less ambitious
audit objective will need to be chosen, such as assessing to what extent the entities´ objectives
have been achieved, target groups have been reached, or the desired level of performance
has been attained.
What is the relationship among the 3Es?

An audit will often focus mainly on one of the 3Es. It is, however, advisable not to examine
aspects of economy, efficiency or effectiveness of activities in total isolation. For example,
looking at economy without also considering the outcome of a policy might lead to inexpensive
but ineffective interventions. Conversely, in an audit of effectiveness, the auditor may also
wish to consider aspects of economy and efficiency. The outcomes of an audited entity,
activity, programme, or operation may have had the desired result, but were the resources
very costly? (GUID 3910/47)
When you use the 3Es in your performance audit, you will often look at more than one area,
and the relationship between them is important to understand. You will often be looking at
10A quasi-experiment studies the impact of an intervention on a target population, but uses methods other than random
assignment to select which members of the population are chosen for participation in the study.
17
resources being used over a given period to achieve an objective or set of objectives. It is
important to understand the relationship between the intervention and its objectives, inputs,
processes, outputs, and outcomes, including results and impacts. Figure 1 explains the
relationship between the 3Es with regard to inputs, outputs and outcomes. It can be helpful
to use and apply this model to the object of your performance audit.
Figure 1: Relationship among the 3Es
Source: Adapted from European Court of Auditors
How does performance auditing promote accountability and transparency?

Performance auditing promotes accountability by helping those with governance and oversight
responsibilities understand the actions needed to improve performance. It can bring to light
hidden issues or problems by examining if decisions by the legislature or the executive are
economically, efficiently and effectively prepared and implemented and whether taxpayers and
citizens have received value for money (GUID 3910/9). It does not question the intentions and
decisions of the legislature or policy but examines whether any shortcomings in their
implementation have prevented the specified objectives from being achieved. (ISSAI 300/12)
Performance auditing promotes transparency by giving an insight into the management and
outcomes of different government activities. The outputs of this work will be of interest to:
• government and legislative bodies;
• taxpayers and other sources of public finance;
• those targeted by government policies; and
• in some cases, the media.
18
Thus, performance auditing directly contributes to providing useful information to citizens while
also serving as a basis for learning and improving the public sector. (ISSAI 300/12, GUID 3910/9)
Performance audits also help the legislature hold government accountable for performance. A
performance audit is often addressed to the legislature, although some countries may have
different arrangements. Depending on the constitutional arrangements in each country, the SAI’s
report may well be the basis of further discussion or hearings at the legislature. In this way,
performance audits promote both accountability and transparency.
What value do performance audits bring?

Performance auditing focuses on areas in which it can add value for citizens and which have the
greatest potential for improvement and provides constructive recommendations for the audited
entities to take appropriate action to improve performance. (ISSAI 300/12)
What value do performance audits add?
Public sector auditing, as championed by the SAI, is an important factor in making a difference to the
lives of citizens. The auditing of government and public sector entities by SAIs has a positive impact on
trust in society because it focuses the minds of the custodians of public resources on how well they use
those resources. Such awareness supports desirable values and underpins accountability mechanisms,
which in turn leads to improved decisions.
Once SAIs’ audit results have been made public, citizens can hold the custodians of public resources
accountable. In this way, SAIs promote the efficiency, accountability, effectiveness and transparency of
public administration.
Source: INTOSAI-P-12
Generally, performance audit offers benefits such as identifying:

• waste and inefficiency in delivering public services;
• opportunities to maximise return on investment in public services;
• risks to the achievement of policy goals; and
• matters of social and economic concern to citizens.
INTOSAI-P-12 explains ways in which SAIs can make a difference in the lives of citizens. Figure 2
shows the specific contributions that performance auditing can make.
19
Figure 2: How performance audits can add value
Performance audit Relevant INTOSAI-P-12 principle How might the Supreme Audit Institution
activity that adds value (SAI) perform this activity?
Integrity 2 - Carrying out audits to ensure that Examining whether government financial
Providing independent government and public sector entities intervention in the housing market has
assurance on success are held accountable for their encouraged buyers who would not have
claimed by government stewardship over, and use of, public otherwise entered the market.
resources. Help to Buy: Equity Loan scheme – progress
review. National Audit Office (UK), 2019.
Accountability 2 - Carrying out audits to ensure that Assessing whether government negotiates a
Helping to hold the government and public sector entities good deal when purchasing medical equipment.
executive to account for are held accountable for their
stewardship over, and use of, public Performance audit report on procurement of
its performance
resources. medical equipment and surgical instruments by
the Department of Clinical Services. Office of the
Auditor General Botswana, 2012.
Transparency 4 - Reporting on audit results and Publishing regional performance data that had
By publishing new thereby enabling the public to hold only been available internally.
information, the SAI can government and public sector entities
shine a light on how public accountable for performance. NHS waiting times for elective and cancer
resources are used treatment. National Audit Office (UK), 2019.
New insights 7 - Being a credible source of Using multiple regression analysis to see which
Applying analytical independent and objective insight and factors have a statistically significant effect on
guidance to support beneficial employee performance.
techniques that have not
yet been used by change in the public sector.
Federal Workforce: Additional Analysis and
government Sharing of Promising Practices Could Improve
Employee Engagement and Performance.
Government Accountability Office, 2015.
Sharing best practice from 7 - Being a credible source of Comparing how different countries manage
home and abroad independent and objective insight and the same activity.
Offering insight based on guidance to support beneficial
experience of auditing similar change in the public sector. Healthcare across the UK: A comparison of the
activities in other NHS in England, Scotland, Wales and Northern
departments. SAIs may Ireland. National Audit Office (UK), 2012.
analyse their individual audit
reports to identify themes, L’accès des jeunes à l’emploi : construire des
common findings, trends, root parcours, adapter les aides (Employment
causes and audit access for young people – building pathways,
recommendations, and adapting state support), Cour des comptes.
discuss these with key (French Court of Auditors), 2016.
stakeholders. SAIs may also
use their engagement in the
international public-sector
auditing profession to draw
lessons from other countries
Making practical 3 - Enabling those charged with public Assessing the root causes of shortfalls in
recommendations sector governance to discharge their performance, then basing their
Including recommendations responsibilities in responding to audit recommendations on this evidence to suggest
in performance audit reports findings and recommendations and how to perform better.
that enable the audited taking appropriate corrective action.
entity to improve its
performance
Clarifying complexity 4 - Reporting on audit results and Writing performance audit publications in a
Providing an easy-to-digest thereby enabling the public to hold simple and clear manner, using language that
summary of complex topics government and public sector entities is understood by all intended users.
accountable.
20
It is important for you as the auditor to think early about whether and how you can aim to provide
value through your performance audit. These considerations will help you design methods,
analyses and communication strategies that maximise the impact of your work.
What types of reports result from performance audits?

The objectives of performance audits – promoting the 3Es and addressing accountability and
transparency – mean that the potential scope of a performance audit may be wide. However,
some themes appear more frequently than others. Figure 3 illustrates some of the common
themes you will likely find in performance audits.
21
Figure 3: Themes that appear in performance audits
Example of a Supreme Audit
Theme Example of an audit objective Institution (SAI) report addressing
this theme
Assessing the extent to which the actions Jamaica's Preparedness for
implemented by the Government of Implementation of Sustainable
Preparedness Jamaica at the national level, since the Development Goals (SDG). Auditor´s
for endorsement of the 2030 Agenda in General Department of Jamaica, 2018.
implementation September 2015, are adequate to
of SDGs support preparedness for the
achievement of the SDGs.
Determining whether a Health Ministry Performance audit report on

uses effective procurement procedures procurement of medical equipment and
Effective to obtain medical consumables or surgical instruments by the Department
equipment at reasonable quality and of Clinical Services of the Ministry of
procurement
price when compared with other Health. Office of the Auditor General
countries. (OAG) Botswana, 2012.
Determining how well public bodies are Green public procurement – is

working together to address the management effectively helping to
Coordination environmental challenges posed by the achieve the climate objective?
across changing climate. Swedish National Audit Office (SNAO),
government 2012.
Assessing whether government support Has European Regional Development

and training for small businesses has led Fund support to small- and medium-
Economic to economic growth, and whether the sized enterprises in e-commerce been
outcomes benefits have been shared equitably. effective?
European Court of Auditors, 2014.
Examining the extent to which gambling Gambling regulation: problem

regulation effectively and gambling and protecting vulnerable
Regulation proportionately protects people from people. NAO UK, 2020.
gambling-related harms and addresses
emerging risks.
Assessing whether the responsible The management of water distribution

ministry is adequately addressing the in urban areas. NAO Tanzania, 2012.
Social issue of leakages in the domestic water
outcomes supply network.
Assessing whether the responsible Sustainable management of fish

ministry is effectively promoting resources in natural waters. OAG
Environmental sustainable management of fish Zambia, 2015.
and resources.
sustainability
outcomes
22
Assessing whether the implementation Elimination of violence against women.
of Women’s Plan of Action, in particular SAI Fiji, 2019.
Gender on elimination of violence against
equality women, is effective by examining: the
existing legal and policy framework;
the process by which the framework
has been implemented; the monitoring
and reporting arrangements over the
implementation of the framework, and
whether improvements can be
demonstrated.
Assessing whether the government,

RADA’s management of the
through Rural Agricultural Development
rehabilitation of farm roads. SAI
Authority (RADA), had in place an
Infrastructure Jamaica, 2019.
effective management system for the
rehabilitation of Jamaica’s farm road
works. Further, whether RADA was
working to maximise adherence to
excellence through the practice of
quality standards in the rehabilitation/
maintenance of roads and minimise
the risk of poor quality of road works.
Determine the extent to which the GASTPE. SAI Philippines, 2018.

Government Assistance to Students and
Teachers in Private Education (GASTPE)
Education Programme achieved its goals and
objectives; whether the Department of
Education ensured the neediest were
prioritised and ensured proper
administration of the programme.
Determining whether the services Prevention services on pregnancy of

provided to prevent pregnancy among adolescents. The services are provided
Health, adolescents are effective. by Ministers of Women and Vulnerable
Population, Health and Education. SAI
education and
Peru, 2018.
gender equality
Source: IDI/PAS Development Team
What is the difference between performance audit and other types of public
audits?
Performance auditing is a specific discipline with its own standards and conventions. It is
important to understand the differences between performance auditing and the other two main
types of public sector audits: financial audits and compliance audits.
It is usually easy to distinguish a financial audit from a performance audit. A financial audit
involves determining, through the collection of audit evidence, whether an entity´s financial
23
information is presented in its financial statements following the financial reporting and
regulatory framework applicable (ISSAI 200/7). SAIs conduct financial audit annually, in which
auditors certify an audited entity’s financial statements. A financial audit adds value by providing
the intended users of the financial statements with confidence in the reliability and relevance of
information presented in the audited statements.
It can be more challenging to understand the difference between a compliance audit and a
performance audit because they sometimes overlap. Compliance audits cover a broad spectrum
of audits, with different characteristics, examining activities, financial transactions or
information.
Compliance auditing is the independent assessment of whether a given subject matter complies
with applicable authorities identified as criteria. Compliance audits are carried out by assessing
whether activities, financial transactions and information comply, in all material respects, with
the authorities which govern the audited entity (ISSAI 400/12). These authorities may include
rules, laws and regulations, budgetary resolutions, policy, established codes, agreed terms or the
general principles governing sound public-sector financial management and the conduct of public
officials. (ISSAI 400/29)
Some performance audits can include compliance questions to the extent that these are
necessary and relevant to examining 3Es of the subject matter.
A performance audit is a direct reporting engagement (ISSAI 100/29-30). In direct reporting
engagements, the auditor selects the subject matter and criteria and measures or evaluates the
subject matter against the criteria, considering risk and materiality. The outcome of the measure
is presented in the audit report in findings, conclusions, recommendations, or an opinion. (ISSAI
100/29)
The other type of engagement is attestation engagement, where the responsible party measures
the subject matter against the criteria and presents the subject matter information. The auditor
gathers sufficient and appropriate audit evidence to provide a reasonable basis for expressing a
conclusion. Financial audits are always attestation engagements, and compliance audits may be
attestation or direct reporting engagements, or both at once. (ISSAI 100/29-30)
24
How does being classified as a direct reporting engagement influence the conduct of
performance audits?
As performance auditing is a direct reporting engagement, it will be part of your role as auditor
to select and define the subject matter of your report and conclusion. It is also part of your role
to identify the relevant criteria, and it will be your task to measure or evaluate the subject matter
against these criteria in order to elaborate an audit report that provides relevant and reliable
information to the users of your audit. You will have a much more active role in asking the
relevant audit questions and in selecting and applying the methods that are relevant for obtaining
audit evidence for the subject matter.
A performance audit may include some checking of the procedures of the audited body, but you
should make sure that the whole audit does not just become a ‘box-ticking’ exercise. Testing
procedures to identify gaps in them does not provide the necessary understanding for assessing
performance. Measuring performance is the process of assessing what the audited entities do to
implement policies. In doing so, you may well need to explain how the procedures you are
checking contribute to a successful outcome. For example, a performance audit assessing how a
Ministry procures vehicles for official use might check that staff follow procurement procedures.
However, it would go on to collect evidence on outcomes, such as:
• How often are the vehicles left unused?
• Did the Ministry pay a fair price for the vehicles?
• Are private businesses able to acquire vehicles more cheaply than the Ministry?
• How can the Ministry reduce the costs of maintaining its vehicles?
• Would it be more cost effective to hire vehicles as and when they are needed?
In a direct reporting engagement, the onus is on you, the auditor, to communicate to the reader:
• what the objective(s) of the performance audit is (are);
• what criteria you have chosen, and why;
• what evidence you have gathered;
• what strengths and weaknesses exist in performance;
• what has caused the weaknesses and why;
• how compelling the evidence is;
• what conclusion you have reached and why;
• what is the impact or consequence of the finding reported; and
• how much assurance the reader can place on the conclusion.
25
Who are the three parties in a performance audit?
The Standard
The auditor shall explicitly identify the intended users and the responsible parties of the audit and
throughout the audit consider the implication of these roles in order to conduct the audit accordingly.
The three parties in public-sector audits are the auditor, responsible party and intended users.
They may assume distinct characteristics in performance auditing.
The auditor's role is fulfilled by the Head of the SAI and by persons to whom the task of
conducting the audits is delegated (ISSAI 100/25). This definition elapses from the different SAI
models. In the Westminster model, the SAI is usually called National Audit Office and the reports
are signed only by the Auditor General, who takes responsibility for the audit. In the Court model
and Board (or Collegiate) model, auditors conduct audits under the supervision of management
level. Thus, the rules have to be interpreted according to these institutional designs (TCU, 2020).
Auditors in performance audits typically work in a team offering different and complementary
skills (ISSAI 300/16).
The responsible party may refer to those responsible for the subject matter, for providing the
auditor with information, and also for addressing the recommendations. In performance audits,
this role may be shared by individuals or organisations. A responsible party may also be an
intended user, but it will typically not be the only one (ISSAI 100/25; ISSAI 300/17; ISSAI 3000/27).
Intended users are the individuals, organisations or classes thereof for whom the auditor
prepares the audit report. The legislature, executive, government agencies, third parties
concerned by the audit, and the public are examples of intended users. (ISSAI 100/25; ISSAI
3000/26)
It is important that you, the auditor, consider the needs and interests of the intended users and
responsible parties. It will help the audit report to add value and to be understandable to these
entities. However, this should not undermine your independence and objective attitude
throughout the audit. (ISSAI 3000/28)
26
What is subject matter and subject matter information?
The Standard
The auditor shall identify the subject matter of a performance audit.
Subject matter refers to the information, condition or activity that is measured or evaluated
against certain criteria. The subject matter relates to the question ‘what is audited’ and is defined
in the audit scope, which is the boundary of the audit. The subject matter of a performance audit
may be programmes, undertakings, systems, entities or funds. They may comprise activities (with
their outputs, outcomes and impacts) or existing situations, including causes and consequences.
The subject matter is determined by the audit objective and formulated in the audit questions.
(ISSAI 100/26; ISSAI 300/19; ISSAI 3000/30)
Subject matter information refers to the outcome of evaluating or measuring the subject matter
against the criteria (ISSAI 100/28). In performance audit, it is the auditor who produces the
subject matter information. It is different in a financial audit, where the responsible party
presents the subject matter information (the financial statements). The auditor then obtains
audit evidence to support an opinion. (TCU, 2020)
27
Chapter 2
What are the key principles of quality performance audits?
This chapter will discuss the eight principles that are necessary for conducting a quality
performance audit. According to ISSAI 100/36-43, these principles are :
• quality control;
• independence and ethics;
• professional judgement and scepticism;
• audit team competence;
• materiality;
• audit documentation and audit supervision;
• audit risk and assurance; and
• communication with audited entities, external stakeholders, media and the public.
Given the focus and nature of performance auditing, these principles are critically important to
SAIs and you as an auditor. Without these principles, SAIs and auditors will not be well-positioned
to effectively execute performance audits and thus achieve improvements in economy, efficiency
and effectiveness (the 3Es). It is important that your SAI has policies and procedures in place that
explain the requirements related to each of these principles. It is your responsibility to follow
them. This chapter touches briefly on SAI-level policies that need to be in place to implement
these concepts’ principles. Still, it is mostly focused on how you, the auditor, can ensure you are
taking the appropriate steps to follow them.
The chapter also has a section on IDI’s considerations to mainstream inclusiveness and maximise
the impact of performance audits.
28
What is quality control?
The Standard
SAIs should establish and maintain appropriate procedures for ethics and quality control.
An SAI’s quality control policies and procedures should comply with professional standards, the
aim being to ensure that audits are conducted at a consistently high level. Auditors should
perform the audit following professional standards on quality control. (ISSAI 100 and ISSAI 140)
SAIs should be consistently focused on delivering high-quality audits and other work. The quality
of work performed by SAls affect their reputation and credibility, and ultimately their ability to
fulfil their mandate (ISSAI 140).
Quality control is a system of policies and procedures put in place by an SAI to ensure that
the audit reports are appropriate, balanced, fair, add value, and are following ISSAIs. Quality
control should be present in all phases of the audit process: planning, execution, reporting,
and follow-up. Such policies and procedures should be set by the head of the SAI, who retains
overall responsibility for the system of quality control (ISSAI 140).
Quality assurance refers to establishing a monitoring process designed to provide the SAI with
reasonable assurance that the policies and procedures relating to the system of quality control
are relevant, adequate, and operating effectively in practice (ISSAI 140/6). The purpose of quality
assurance is to conduct the review to ascertain if the audit was conducted following ISSAIs.
Reviews, procedures and checks taking place before the report is issued are part of the SAIs
quality control system, to ensure that the reports are of high quality. These reviews can be done
by managers or external reviewers. Quality assurance, on the other hand, involves checking if the
appropriate quality control systems have been put in place and if they are appropriately
implemented. It is not done by line managers and includes reviewing already published reports.
(AFROSAI-E PA Handbook, 2016)
The six key elements of quality control are shown in Figure 4. Additionally, Appendix 1 provides
an example of an SAI quality assurance framework.
29
Figure 4: Key elements of quality control for performance auditing
Tone at the top of the Supreme Audit

Institution (SAI) and its policies should
Leadership promote a culture that focuses on quality.
Audit work and reports

are assessed against a Ethical
system of quality control. Monitoring All auditors working for an
requirements
This can be considered SAI should follow certain
quality assurance. standards for ethical
behaviour.
Audit work should adhere to SAIs should only carry out

all requirements that apply; Acceptance audits where it is
legal, regulatory, and Engagement
policies. performance and competent to perform the
continuance work, has the capabilities,
can comply with relevant
Human
ethical requirements and
resources has considered how to
The SAI should have the resources treat risks to quality that
available to carry out its work in a way might arise.
that meets all standards.
Source: ISSAI 140
For a system of quality control to be effective, it needs to be part of an SAI’s strategy, culture,
policies and procedures. In this way, quality is built into the performance of the SAI’s work and
the production of the SAI’s reports, rather than being an additional process once a report is
produced (ISSA1 140). Quality control procedures should cover matters such as the direction,
review and supervision of the audit process and the need for consultation to reach decisions on
difficult or contentious matters. (ISSAI 100/38)
It is not enough that an SAI puts policies and procedures in place; the functioning of the quality
control system needs to be monitored through a regular assessment of audit work and reports.
This is important to determine whether the system is suitably designed and operating effectively
and if policies and procedures are being followed. This monitoring can be conducted through
both internal and external reviews. Monitoring, such as periodic peer reviews or other types of
review activities, helps SAIs assure that the work performed and the resulting reports meet
standards and are of high quality.
Your SAI needs policies and procedures that codify the actions and behaviours expected of you
according to each element of a quality control framework. The SAI should ensure these are clearly
communicated to all auditors.
30
What are independence and ethics?
Adhering to independence and ethical requirements is a prerequisite to conducting performance

audits and is emphasised in the INTOSAI Framework of Professional Pronouncements (IFPP). The
following sections present important concepts regarding independence and ethics.
Independence
Independence means being free from circumstances or influences that

INTOSAI’s ISSAI 130: compromise, or maybe seen as compromising, professional judgement,
Code of Ethics provides a and acting in an impartial and unbiased manner.
detailed discussion of
independence and ethics. The
code provides SAIs and the It is important that your SAI and you, the auditor, understand what
staff working for them with a potential threats exist that could affect independence and undermine the
set of values and principles on
which to base behaviour. effectiveness of an audit. There are six major threats to independence
Furthermore, the code, during a performance audit, as shown in Figure 5.
recognising the specific
environment of public-sector
auditing (often different from
that of private sector auditing),
gives additional guidance on
how to embed those values in
an SAI’s processes and audit
work.
31
Figure 5: Six major threats to independence during a performance audit
Self-interest The threat that a financial or other personal interest will inappropriately influence
an auditor’s judgement or behaviour.
Bias or The threat that an auditor will, as a result of political, ideological, social or other
advocacy convictions, take a position that is not objective.
Familiarity The threat that personal relationships with family, friends, etc. in the audited
agency will cause the auditor to take a position that is not objective.
Intimidation The threat that external influences or pressures will impact an auditor’s ability to
or undue make independent and objective judgements.
influence
Self-review The threat that an auditor or audit organisation that has provided services to the
audited agency will use that experience to affect its conclusions.
Management The threat that results from an auditor crossing the line from being an external
participation auditor to being a part of the internal management structure of the audited
agency.
Source: GUID 3910/14
SAIs and audit teams should apply control mechanisms that eliminate or reduce a threat to
independence to an acceptable level, such as those listed in the box below.
SAIs should also ensure their personnel do not develop too close of a relationship with the
entities they audit, so they can remain objective. SAIs, while adhering to the laws enacted by the
legislature that apply to them, should also be free from direction or interference from their
legislature or government in the:
• selection of audit topics, if applicable, as some SAIs must perform audits of certain topics
based on their mandate. Regardless, it is important that the SAI and auditor maintain
independence in conducting audits;
• planning, programming, conducting, reporting and following-up of their audit;
• organisation and management of their office; and
• enforcement of their decisions where the application of sanctions is part of their mandate.
32
As an auditor, it is important to remain independent so that your report Auditors can maintain
will be impartial and be seen as such by the intended users. Your ability independence by:
to maintain independence is important in the context of a performance • avoiding participating in audits
audit, as many decisions must be made based on your professional in which the auditor has a
judgement and audit evidence. financial or personal interest;
• not accepting gifts, bribes or

Common decisions made by the auditor (which are discussed in later other payments that would
chapters) in which a lack of independence could negatively affect a create the appearance of bias or
advocacy;
performance audit include:
• recusing themselves of audits in
• identifying and deciding on an audit topic; which family or friends have
• establishing the audit objective(s) and questions; invested interests in the outcome
of the audit;
• identifying the applicable criteria;
• avoiding involvement in
• determining the methodological approach to the audit; external activities that could be
• assessing audit evidence and forming conclusions; seen as having an undue
influence on an audit’s
• deciding on audit criteria and findings, after independently assessing outcomes; and
them and discussing them with the entity or entities that are the
• avoiding becoming too close to
focus of the audit; the audited entity and its
• assessing the positions of various stakeholders; and management, resulting in bias or
advocacy.
• writing a fair and balanced report.
What are some control mechanisms that can safeguard against threats
to independence?
✓ Involve another person to review the work ✓ Ensure that all individuals working on an audit
done or advise as necessary without confirm their independence before
compromising the auditor’s independence. commencing work on the audit and consider
their independence throughout the audit.
✓ Consult a third party, such as a committee of
independent directors, a professional ✓ Remove a person from the audit team when
regulatory body, or a professional colleague. that person’s financial interests, relationships, or
activities threaten independence.
✓ Rotate personnel to performance audits of
different entities after a few years to counter
the familiarity threat.
33
Ethics
Ethics are the moral principles of an individual that include integrity,

It is important that an
professional competence and due care, professional behaviour and auditor remains free of
confidentiality, as defined below: conflicts of interests, both real
and perceived, and maintains
• Integrity: To act honestly, reliably, in good faith and the public independence of mind and
independence in appearance.
interest.
• Professional competence: To acquire and maintain knowledge and Accordingly, an auditor must
avoid both conflicts and the
skills appropriate for the role and act in accordance with applicable appearance of a conflict or any
standards and with due care. other situation where an
objective, knowledgeable third
• Professional behaviour: To comply with applicable laws, regulations party might have reason to
and conventions, and avoid any conduct that may discredit your SAI. question whether the auditor can
make objective and impartial
• Confidentiality and transparency: To appropriately protect judgements.
information, balancing this with the need for transparency and
accountability. Source: IDI/PAS Development Team
Each of these principles is discussed in more detail below. INTOSAI’s ISSAI 130: Code of Ethics
provides in-depth guidance for both the SAI and the auditor regarding each of these principles.
SAIs should have policies and procedures that address ethical requirements and emphasise the
need for compliance by each auditor. Ethical requirements of the SAI has to include requirements
set down in legal and regulatory frameworks that govern the SAI. SAIs need to consider: written
declarations from personnel to confirm compliance with the SAI’s ethical requirements; and to
put procedures in place for personnel to report breaches of ethical requirements.
Integrity
SAIs should have policies and procedures in place that:

• emphasise, demonstrate, support and promote integrity;
• ensure the internal environment is conducive for staff to raise ethical breaches; and
• ensure responses to integrity breaches are taken in a timely and adequate manner.
You, the auditor, need to act honestly. You also need to be alert to circumstances that might
expose you to integrity vulnerabilities and avoid disclosing them as appropriate. These
circumstances may involve:
• personal, financial or other interests or relationships that conflict with the SAI’s interests;
• the offer of gifts or gratuities from the audited entities;
• the opportunity to abuse power for personal gains;
34
• involvement in political activities, or participation in pressure groups, lobbying, etc.;
• access to sensitive and confidential information; and
• the use of SAI resources for personal or other purposes.
Professional competence
SAIs need to adopt policies and procedures to ensure performance Auditors exhibiting
professional competence
audits and related tasks are conducted by staff with the appropriate and behaviour are
knowledge, skills and abilities to successfully conduct their work. Such important to the execution of
performance audits. For example,
policies and procedures can include: auditors have to:
• putting in place competence-based recruitment and human
• be objective, neutral, non-
resources practices; partisan, and fact-based;
• assigning work teams that collectively possess the expertise required
• use methodologically sound
for each assignment; approaches to address audit
• providing staff with appropriate training, support and supervision; objectives; and
• providing tools to enhance knowledge and information sharing, and • be able to effectively apply SAI
policies and procedures
encourage staff to use these tools; and
regarding professional behaviour
• addressing challenges arising from changes in the public sector and norms.
environment. Auditors cannot:
In assessing and maintaining professional competence requirements, • select sites to visit as part of the
you, as an auditor, can: audit based on personal reasons;
• understand your role and tasks to be performed; • post personal opinions on social
• know the applicable technical, professional and ethical standards to media about issues relevant to an
ongoing performance audit;
be followed;
• work competently in a variety of contexts and situations, depending • misuse their position to obtain
information for personal use; and
on the requirements of the job or task; and
• acquire new knowledge, skills and abilities, updating and improving • engage in outside activities that
would create a conflict of interest
them as needed. on the part of the auditor or SAI.
35
Professional behaviour
Your SAI should be aware of the standards of professional behaviour expected by its internal and
external stakeholders, as defined by the laws, regulations and conventions of the society in which
they operate, and conduct business accordingly and in line with its mandate. To promote the
highest standards of professional behaviour and to identify activities that are inconsistent with
that standard, SAIs have to provide direction on expected behaviour and implement controls to
monitor, identify and resolve deviations from it.
It is important that you, the auditor, take steps to ensure your behaviour, both within and outside
the working environment, abides by professional norms, such as:
• knowing SAI policies and procedures relating to professional behaviour, such as applicable
professional standards, laws, regulations and conventions of the society in which the SAI
resides;
• understanding the impact of your actions on the SAI’s credibility, and considering how your
behaviour, both within and outside the working environment, might be perceived by
colleagues, family and friends, auditees, the media and others. For example, work or
volunteering you do outside your SAI activities could be seen as a conflict of interest or impact
your impartiality. Some SAIs have a reporting mechanism for reporting outside activities. See
Appendix 2 for an example of an SAI form for documenting participation in outside activities;
• being aware that common expectations include acting according to ethical values, adhering to
the legal and regulatory frameworks in place, not misusing your position, applying diligence
and care in performing your work, and acting appropriately when dealing with others; and
• applying appropriate prudence and care to help ensure your actions or opinions do not
compromise or discredit the SAI and its work, for example, when using social media.
Confidentiality and transparency
SAIs should have policies and procedures to ensure that it balances the confidentiality of audit-
related and other information obtained during a performance audit with the need for
transparency and accountability. The SAI should also have an adequate system in place for
maintaining confidentiality as needed, especially about sensitive data. Further, SAIs should
ensure that any parties contracted to carry out work for the SAI are subject to appropriate
confidentiality agreements.
36
As an auditor, it is important to be aware of any related legal obligations and your SAI’s policies
and procedures concerning confidentiality and transparency. You are also responsible for
protecting the information you collect during the audit and not disclose it to third parties unless
they have proper and specific authority or there is a legal or professional right or duty to do so.
Examples of controls and safeguards you can apply to help ensure confidentiality and
transparency include:
SAIs can take the

• Use professional judgement to respect the confidentiality of
following steps to help information collected as part of an audit. In particular, keep the
ensure transparency:
confidentiality of information in mind when discussing work-related issues
• Adopt audit standards, with other SAI employees.
processes, and methods that
are objective and transparent
to the audited entities. • If you are in doubt about whether suspected breaches of laws or
• Make public the results of regulations have to be disclosed to appropriate authorities or parties,
their work, including their consider obtaining legal advice from within your SAI to determine the
methodology(ies) for
conducting the work. appropriate course of action.
• Communicate timely and

widely on their activities and • Do not discuss information related to your audit outside the work
audit results through the media,
environment, including on social media.
websites and other means.
Source: IDI/PAS Development Team • Secure electronic data devices, such as laptops and portable data
storage devices, and ensure all audit information, such as audit-related documents and
papers, are secured appropriately. You could do this by ensuring that information is stored in
locked areas, such as cabinets or offices, and also by controlling access to the office space to
ensure the protection of all audit-related information, both electronic and hard-copy
documents and papers. For electronic information, steps need to be taken to prevent loss
through backing up data and servers, as appropriate.
Mitigating or resolving independence and ethical concerns
SAIs and auditors have responsibilities to mitigate and resolve independence and ethical
concerns. SAIs should have an ethics control system to identify, analyse and mitigate ethical risks,
support ethical behaviour and address any breach of ethical values, including protecting those
who report suspected wrongdoing. An ethics control system’s main components could be a code
of ethics, leadership and ‘tone at the top’, ethics guidance, and ethics management and
monitoring (for more information on SAI responsibilities, see ISSAI 130).
37
As an auditor, you need to take concrete action to mitigate or resolve independence and ethical
issues, such as by:
• identifying situations where your independence and ethical requirements can be impaired,
and understanding the potential impacts of such situations;
• signing declarations of interests and conflict to help identify and mitigate threats to
independence and ensure both your own integrity and that of the SAI. See Appendix 3 for an
example of an independence statement;
• informing your management about relationships and situations that may present a threat;
• maintaining and developing your knowledge and skills to ensure a full understanding of
behavioural norms and expectations, professional competence, and the protection and
confidentiality of information related to the audit; and
• informing your supervisor if your expertise is not sufficient to perform a specific task to ensure
professional competence and integrity.
You should be aware of your own biases and opinions regarding topics and organisations. Police
your behaviour to ensure you are upholding the independence and ethical requirements.
Consider any independence and ethical threats at many points throughout the planning and
execution of the audit. If you have questions about what might be a threat to ethics and
independence, trust your instincts that there may be an issue and review your SAI’s policy and
raise the issue to your superiors when appropriate. Additionally, be aware of the behaviour of
other auditors and colleagues because the reputation of your SAI rests on all of its auditors
upholding independence and ethical requirements. Many SAIs have procedures for reporting
observed misconduct. If in doubt, check with your supervisor.
What are professional judgement and scepticism?
The Standard
The auditor shall exercise professional judgement and skepticism and consider issues from different
perspectives, maintaining an open and objective attitude to various views and arguments.
Exercising professional judgement and scepticism are critical to ensuring an effective

performance audit.
Professional judgement is the act of applying knowledge, skills, and experience – in a way that is
informed by standards, laws and ethical principles – to develop an opinion or decision on an issue.
Professional scepticism means maintaining a professional distance from the entity or entities
38
being audited and an alert and questioning attitude when assessing the sufficiency and
appropriateness of audit evidence obtained throughout the audit.
SAIs need to have policies and procedures to guide auditors to consistently apply professional
judgement and professional scepticism. For example, using professional judgement is important
to auditors in applying the conceptual framework to determine independence in a given
situation. As such, SAI policies and procedures need to include guidance for identifying and
evaluating any threats to independence, including threats to the appearance of independence
and related safeguards that may mitigate the identified threats.
SAIs should also ensure that auditors understand the importance of professional judgement and
scepticism and can apply it appropriately within a performance audit. To achieve this end, SAIs
could require auditors to participate in periodic training that focuses on, for example:
• the types of evidence – documentary, testimonial, physical and Auditors need to use their
analytical – and their strengths and weaknesses; professional judgement
and scepticism throughout a
performance audit, including in:
• the standards – appropriateness and sufficiency – used in assessing
• developing audit questions that
evidence; and are objective and neutral;
• selecting appropriate scope
and methodologies;
• the importance of corroborating evidence to ensure the • conducting interviews with
conclusions reached by auditors are reasonable and logical. officials;
• assessing the evidence
collected during the audit; and
You, as an auditor, should exercise professional judgement and • developing a message for the
scepticism, consider issues from different perspectives, and maintain written report that is balanced.
an open and objective attitude toward various views and arguments.

This open-mindedness is essential if you are presented with Source: IDI/PAS Development Team
contradictory information that needs to be assessed and considered in conjunction with other
evidence collected during an audit. Exercising professional judgement and scepticism allows you
to be receptive to a variety of views and arguments and be better positioned to consider different
perspectives.
Performance audits require significant judgement, interpretation and scepticism because

evidence associated with performance audits is typically persuasive rather than conclusive,
requiring constant reassessment. Professional judgement and scepticism are key for you to
effectively and critically assess the audit evidence obtained during an audit. Rather than
approaching audit work in a ‘tick the box’ mentality, you must challenge the information and
evidence obtained. You need to step back, look at the wider context and ask, “does that make
sense?”.
39
Some examples of how you can apply professional judgement during performance audits include:
• determining the required level of understanding of the subject matter;

• determining the nature, timing and extent of audit procedures and methodology;
• determining which findings are significant enough to report;
• evaluating whether sufficient and appropriate audit evidence has been obtained; and
• determining the recommendations to be made, as appropriate, to address root causes of the
problems identified.
How might professional scepticism be applied?

✓ Question responses to inquiries and other ✓ Evaluate the reliability of data to be used
information obtained from the audited during the audit.
entity. ✓ Assess the reliability of the source of the
✓ Corroborate testimonial or evidence from a documents obtained.
single source with secondary sources of ✓ Be alert to audit evidence that contradicts
evidence.
other audit evidence obtained.
✓ Revise your risk assessment as a result of
identified material or significant inconsistent
information (discussed later).
Source: GUID 3910/88-89
As an auditor, you can help to improve the strength of the evidence obtained by exercising
professional scepticism (by asking questions to test the accuracy of evidence), following up when
things do not make sense, and not accepting what the audited entities’ management tells you
without corroboration. Professional scepticism is critical to ensuring you can answer the audit
questions and make conclusions with a high level of assurance.
What is audit team competence?
The Standard
The SAI shall ensure that the audit team collectively has the necessary professional competence to
perform the audit.
The auditor shall maintain a high standard of professional behaviour.
Source: ISSAI 3000/63 and ISSAI 3000/75
40
Conducting an effective performance audit requires putting in place
Effective audit teams
include team members
a team that has all the skills needed for carrying out the necessary
that collectively have: tasks required during an audit.
• teamwork and collaboration
skills; The quality of an audit is dependent on the skills, abilities and
• critical thinking skills; knowledge of the audit team. Performance auditing is a team effort.
• quantitative and qualitative
analytical skills; Performance audit issues are often complex, and not all team
• interviewing skills; and members need to possess every needed Ways for auditors to
• oral and written skill. Rather, the audit team has ideally be exhibit professional
communication skills. behaviour during a
comprised of team members with a variety performance audit:
of skills, abilities and knowledge to ensure
• Treat audited entity officials as
it is positioned to carry out the audit work. professionals and with respect.
• Respond promptly to inquiries

SAIs should ensure their audit teams collectively have the necessary from representatives from the
professional knowledge, skills and abilities before performing the audited entity.
audit. For example, SAIs could recruit staff with the appropriate • Be on time for meetings with
qualifications to include areas of study and knowledge of needed officials from the audited entity
and other stakeholders;.
disciplines. Once hired, SAIs can also require or suggest a specific
curriculum of training to ensure their auditors have the necessary skills • Dress professionally and in
accordance with SAI policies.
and abilities. Training can include classroom instruction, individual
• Abide by SAI policies and
study and on-the-job training based on individual needs and the SAI’s procedures in conducting the
curriculum, among other initiatives. Further, a prescribed amount of audit.
continuous learning can be required by an SAI. Auditors can maintain knowledge
of professional behaviour norms
and expectations by:
It is also important for SAIs to ensure that the experience levels of the
auditors, supervisors and managers are appropriate for the audit. For • completing periodic training on
audit processes, procedures, and
example, if there are some inexperienced auditors on the audit team, requirements; and
it is important to balance them with experienced supervisors and
• participating in conferences
managers. A team lacking the necessary skills, abilities, knowledge and and seminars to (1) stay abreast
experience may carry out an audit in a less than efficient and effective of technical and professional
standards and (2) expand
manner and produce a report that does not appropriately address the knowledge in the public policy
audit topics. issue area that the auditor works
in.
Subject matter experts, who are stakeholders either internal to the SAI Source: IDI/PAS Development Team
or contracted by the SAI to assist the audit team, are often used in
performance auditing to complement the skill set of the audit team and to improve the quality
of the audit. For example, stakeholders internal to an SAI could be legal, methodological or
technical experts that are not full-time members of a specific audit team but provide their input
41
and expertise as needed throughout an audit in order to improve the quality of the work. Before
consulting with these stakeholders, the SAI and you, the auditor, should ensure the expert has
the necessary competence required for the audit and that they are informed about the
conditions and ethics surrounding the audit process. This also applies to experts that are not part
of the SAI. See below for examples of areas where different types of
expertise can be useful for a performance auditing team. When an audit is initiated
and the audit team is
being assembled, SAIs can:
Once the audit team has been assembled, and initial stakeholders
• identify the appropriate areas
identified, it is important that all involved maintain a high standard of of expertise within the SAI that are
professional behaviour. You should comply with all legal, regulatory necessary to carry out the audit
as well as the roles and
and professional requirements, and avoid all conduct that might responsibilities for the individuals
discredit your work. Maintain individual professional skills and representing these areas of
expertise for the audit; and
competence by keeping abreast of, and complying with, developments
in professional standards and pertinent legislation. It is important that • assign staff to the audit team
that ensures the necessary skills,
all these professional behaviours are maintained throughout the audit abilities and knowledge across
process, from topic selection and audit planning through data the members of the audit team to
effectively conduct the audit.
collection, analysis, reporting and follow-up. These commitments help
ensure that a quality audit is conducted. Source: IDI/PAS Development Team
In addition to maintaining a high standard of professional behaviour, ISSAI standards state that
auditors have to also be willing to innovate throughout the audit process, such as by being willing
to suggest or try new methods or ideas. By being creative, flexible and resourceful, you will be in
a better position to identify opportunities to develop innovative audit approaches for collecting,
interpreting and analysing information.
What types of expertise can be useful for performance auditing team

members, project managers or outside experts?
✓ Research design. ✓ Organisational management.
✓ Scientific evaluation methods. ✓ Specialised expertise depending on the topic of

the audit, such as information technology,
✓ Legal.
cyberspace or engineering.
✓ Social sciences.
42
What is materiality?
The Standard
The auditor shall consider materiality at all stages of the audit process, including the financial, social and
political aspects of the subject matter with the goal of delivering as much added value as possible.
Materiality can be defined as the relative importance (or significance) of a matter within the
context in which it is considered (ISSAI 3000/84). It can influence the decisions of users of the
report, such as legislatures or executives, to deliver as much added value as possible. In addition
to monetary value, materiality includes social and political significance, compliance,
transparency, governance and accountability. It is important for the auditor to keep in mind that
materiality can vary and can depend on the perspective of the intended users and responsible
parties.
The inherent characteristics of an item or group of items may render a matter material by its very
nature. A matter may also be material because of the context in which it occurs. Materiality
considerations affect decisions concerning the nature, timing and extent of audit procedures and
the evaluation of audit results. Considerations may include stakeholder concerns, public interest,
regulatory requirements and consequences for society. The selection of audit topics and the
audit itself needs to consider the concept of materiality.
Another consideration in determining materiality is inclusiveness. Inclusiveness is the practice or

policy of including people who may otherwise be excluded or marginalised, such as those with
physical or mental disabilities and members of minority groups. It is essential for governments to
ensure social inclusiveness, equal opportunity and equity. Given this, public auditors need to
consider inclusiveness as a dimension in the audit. See the end of this chapter for more
information on inclusiveness.
The principle of materiality has to be included in SAI policies and procedures guiding all aspects
of performance audits. Specifically, materiality needs to be considered in selecting audit topics,
identifying and defining criteria for the audit, evaluating audit evidence and documentation, and
managing the risks of producing inappropriate or low-impact audit findings or reports.
Ultimately, the auditor’s consideration of materiality requires the application of professional

judgement. Specifically, it is up to you and your audit team to distinguish the material and
43
immaterial. ISSAI 3000 identifies concepts to be considered in making decisions related to
materiality when selecting audit topics, such as:
• indications or risks to economy, efficiency and effectiveness;

• financial significance, socially and politically;
• maximising the expected impact;
• auditability; and
• falling within the SAIs mandate.
As an auditor, keep materiality in mind throughout the audit, such as when designing audit
questions and criteria, when collecting and assessing evidence associated with the audit, and
formulating audit findings and recommendations that significantly contribute to improved
performance. For example, the entirety of audited entities’ operations is more than likely not
material to your audit, so you should concentrate your effort on the topic that is material and
the focus of the audit. You could spend immeasurable time collecting documents about a topic,
but to make the best use of available resources, always consider the materiality of a document
or discussion when conducting the work. The next chapters provide more detail about the
principle of materiality as it pertains to all phases of the audit process.
What are audit documentation and audit supervision?

This section describes the importance of audit documentation and audit supervision. While these
topics are introduced below, they are discussed in detail throughout this handbook as we discuss
the various phases of a performance audit.
Audit documentation
The Standard
The auditor shall document the audit in a sufficiently complete and detailed manner.
Audit documentation records audit procedures performed, relevant audit evidence obtained,
and conclusions the auditor reached (terms such as ‘working papers’ or ‘audit trail’ are also
sometimes used).
44
What should an experienced auditor be able to understand from audit documentation?
✓ The nature, time and extent of the work ✓ The conclusions reached as a result of the
conducted. aforementioned significant matters.
✓ The findings of the audit work and the ✓ Significant or key decisions made in reaching
evidence obtained. those conclusions.
✓ Significant matters that arose during the audit
(for example, changes in the scope or
approach of the audit, decisions regarding a
new risk factor identified during the audit,
actions taken as a result of disagreement
between the audited entity and the team,
etc.).
It is important that SAIs have policies and procedures that define the basic standards of audit
documentation required for audits performed by the SAI. These policies and procedures define
the standards for the types of files that must be maintained and for how long once the audit is
completed. SAIs should provide training to auditors regarding how documentation for audits will
be compiled and maintained. The policies, procedures and training help ensure that audit
documentation collected for each audit provides evidence of the auditor’s basis for a conclusion
about achieving the overall objective(s) of the audit. The policies, procedures and training also
aim to help prove evidence that the audit was planned and performed in accordance with SAI’s
requirements and applicable legal and regulatory requirements.
As an auditor, you should take steps throughout the audit to ensure that proper audit
documentation is being collected and maintained according to SAI policy. You also need to ensure
that the documentation collected is sufficient to enable an experienced auditor, having no
previous connection with the audit, to understand decisions made and how the audit results
were obtained. Documentation starts at the very beginning of a performance audit when the
audit team is first assembled. You will need to consider documenting the following as you begin
your audit:
• How the audit topic was selected.

• Any pre-planning that was conducted.
• How stakeholders were identified.
• Any communication with the audit entities.
45
• Any initial decisions by the team and management. There are many
• Any risks that were identified. types of documents that
auditors need to maintain as part
of the audit documentation.
Documentation will continue to be very important as you move to These documents include, but
conduct and report on the audit, and documentation should be are not limited to:
completed promptly. You should document: • an index for the information

included in the file;
• the evidence and your team’s analysis of that evidence; • a memo of all key decisions
• how you arrived at the findings; and and communications that
documents the decisions,
• internal reviews, communication with the audited entities and the performed activities, and internal
and external communication
considerations for making (or not) changes based on comments during the audit;
received and other key decisions made as you develop a message and • statements of independence
for the auditors;
draft report. • the audit plan that identifies
the scope, methodology and
It is important your audit team begins its audit documentation set at the plans to collect the evidence for
the audit;
very beginning of the audit in order that all information collected and • documentation of meetings
with the audited entity, such as
decisions made are properly documented. The audit team needs to reach the initial meeting to discuss the
an agreement about the organisation of the audit documentation as well audit and meetings near the end
of the audit during which findings
as any processes and approaches that will be used by the audit team to are discussed;
document the audit. For example, many types of documents in an audit • interview records to document
testimonial evidence from
are easier to organise and manage when filed electronically in a clear officials;
• evidentiary documentation,
folder structure for the project. To the extent allowed by your SAI policy such as the audited entity’s
and procedures, electronic documentation can replace physical copies of policies and procedures, memos,
briefings on the programme, and
documents provided the electronic documentation is sufficiently backed data;
up. Some evidence collected as part of the audit may not be able to be • records of file reviews,
observations or inspections;
stored electronically but still needs to be saved and stored to ensure the • answers to questionnaires;
• records of discussions during
proper documentation of the audit. For example, you may collect physical focus groups and reference
objects as part of the audit that cannot be stored electronically in a groups;
• records of analyses;
computer-based storage system. These physical objects have to be • documentation of key
stakeholder and management
maintained as part of the audit documentation set. Lastly, follow your reviews of drafts reports; and
SAI´s guidance for keeping audit documentation for an adequate period • the final report cross-
referenced to the evidence in
of time. the file, either by notes written in
a copy of the final report or in a
separate document explaining
Your audit team could consider establishing an approach to cross- the evidence that was used for
the main findings and where this
reference the evidence collected throughout the audit with the team’s evidence is available in the file.
analysis. Alongside collecting and cross-referencing documentation, you
can ensure that proper procedures are used to maintain the Source: IDI/PAS Development Team
confidentiality and safe custody of documentation and working papers.
46
Below is a sample of a basic electronic file structure for a performance audit. The structure can
be adapted based on the needs of the team and SAI policy.
Audit documentation folders

• Administrative documents.
• Background materials.
• Planning materials.
• Evidentiary materials (for example, interview records, documents obtained).
• Analyses of evidence.
• Draft reports.
• Follow-up.
Audit supervision
The Standard
The SAI shall ensure that the work of the audit staff at each level and audit phase is properly supervised
during the audit process.
Generally, the audit supervisor is responsible for ensuring that all audit policies and procedures
are followed throughout the audit process.
Audit supervision involves providing sufficient support, guidance and direction to staff assigned
to the audit to ensure the audit objective(s) are addressed, methodologies are applied
appropriately, evidence and analysis are sufficiently documented, and the report is of high
quality. Supervisors must stay informed about significant problems encountered during the audit
and continually review the work performed to ensure a quality audit. An important part of audit
supervision is providing effective on-the-job training to members of the audit team so that all
auditors are developing their capacity to carry out audits effectively.
47
What does audit supervision consist of?
✓ Ensuring that all team members fully ✓ Considering the competence and capabilities
understand audit objective(s) and audit of individual members of the engagement
questions. team.
✓ Ensuring that audit procedures are ✓ Addressing and handling significant matters
adequate and properly carried out. that arise during the engagement.
✓ Ensuring that audit evidence is relevant, ✓ Supporting the auditor when needed to
reliable, sufficient and documented. overcome challenges.
✓ Ensuring international and national auditing ✓ Providing hands-on support in solving issues
standards are followed. that arise.
✓ Tracking progress of the engagement to ✓ Identifying matters that require more
ensure that budgets, timetables and experience to review.
schedules are met. ✓ Reviewing and approving the audit work.
It is important that SAIs provide guidance, and supervisors have to provide coaching and review
during all phases of an audit to ensure that the audit:
• complies with professional standards;

• achieves the intent of the audit’s objectives;
• documentation is complete and supports the audit’s findings and recommendations; and
• staff members develop their professional competence.
Some SAIs have a central office that reviews the outputs of all audits for compliance with audit
standards after supervisory review. The central office review ensures, for example, that the
findings, conclusions and recommendations are sufficiently and appropriately supported by
evidence and are clearly presented.
The degree to which supervision is needed depends on multiple factors, such as the size of the
audit organisation, the experience of the auditors and the significance of the work. For example,
an audit involving issues with a high degree of materiality, such as audit topics that require large
amounts of governmental funds for operation or issues that are extremely sensitive from a
political or societal perspective, is likely to necessitate a greater degree of supervision and
oversight within the audit team and the SAI. Regardless of these factors, audit work needs to be
reviewed by a senior member(s) of the audit team and SAI management throughout the audit
process, especially before the audit report is finalised.
As an auditor, ensure that you adhere to your SAI’s requirements regarding supervision. For
example, provide audit documentation to your supervisor for their review and input. You also
have to be receptive and respond appropriately to any direction, coaching, monitoring and
feedback provided by your supervisor, and seek to continuously improve your professional
competence and performance.
48
What are audit risk and assurance?
The Standard
The auditor shall actively manage audit risk to avoid the development of incorrect or incomplete audit
findings, conclusions, and recommendations, providing unbalanced information or failing to add value.
The auditor shall communicate assurance about the outcome of the audit of the subject matter against
criteria in a transparent way.
SAIs and auditors should actively manage audit risk. The management of risk should allow an SAI
and audit team to provide assurance that the intended users can be confident about the
reliability and relevance of the information provided by the audit, and that the results can be
used as the basis for making decisions.
There are numerous risks associated with performance auditing, as shown in Figure 6. The SAI
and its auditors must provide assurance to its users that these risks are appropriately minimised
and managed.
49
Figure 6: Common risks in performance auditing
Incorrect or Auditors reach incorrect or incomplete audit conclusions and make

incomplete recommendations that are not focused on the necessary or appropriate issues.
This can occur as a result of numerous factors, such as an inadequate assessment
conclusions of the evidence and not following appropriate and necessary audit procedures.
Unbalanced Auditors fail to include and evaluate contrary evidence, clearly identify which audit
information criteria are met, or report on good practices. Achievements of the audited entity
are not discussed, and contributing factors to the deficiencies identified are not
disclosed. For example, shortfalls are highlighted without explaining the challenges
or constraints under which the entity operates, or the audited entity’s performance
is assessed without reference to acceptable standards.
No or limited Auditors fail to provide new information or knowledge from the audit.
added value Specifically, the auditors do not add new analytical insights (broader or deeper
to the users as analysis or new perspectives) or make information accessible to various
stakeholders.
a result of the
audit
Difficulties in Auditors do not have access or have limited access to needed information.
obtaining Additionally, the information may not meet quality standards (that is, the data are
not reliable). As the audit findings and conclusions rely greatly on the quality of
quality information and data collected, it is essential to assess the risk of not having access
information to good-quality information and data when planning and conducting an audit.
Insufficient Auditors do not conduct sufficient analysis due to the lack of expertise, audit
analysis criteria or access to information. If due care is exercised during the planning stage,
risks due to a lack of expertise and audit criteria can be mitigated during the audit.
Omission of Auditors do not identify all of the key issues at the planning stage that will be
relevant covered during the audit, fail to consider relevant pieces of evidence or fail to
information or counter important arguments in the audit’s conclusions.
arguments
Presence of
Auditors do not assess whether the risk of fraud is significant within the context of
fraud, abuse of the audit objective(s) and/or fail to communicate fraud and irregularities promptly.
resources If fraud exists, the auditor is encouraged to follow SAI procedures regarding fraud.
and/or irregular
practices
Substantial Auditors do not appropriately handle highly complex and politically sensitive
complexity or topics. This could seriously undermine the credibility of the audit report and the SAI.
political
sensitivities
50
The concept of audit assurance is inseparable from the concept of
audit risk. Performance auditors are not normally expected to provide Audit teams and auditors
have to strive to manage
assurance as an overall opinion, comparable to the opinion on financial audit risk and provide assurance
audits, on the audited entities’ achievement of economy, efficiency by ensuring:
and effectiveness (ISSAI 300/21). • the audit is within the SAI’s

scope of authority, and the
scope is defined in an
The users of an audit report should be confident that the report appropriate manner;
conclusion is reliable and valid. According to GUID 3910/27, reliable
• the human capital with the
and valid information requires that the conclusions on the subject necessary and appropriate skills
matter are logically linked to the audit objective(s) and criteria and are and knowledge are available
within the auditing entity to
supported by sufficient and appropriate evidence. The conclusions execute the audits;
must be written in a way that enhances the degree of confidence of
• the design of the audit
the intended users about the evaluation of the underlying subject effectively accounts for and
manages risks;
matter against criteria.
• there is access to the records
and data needed to perform the
It is important for the auditor to make the links clear between the audit audit;
objectives, criteria, and findings based on solid evidence. This is done
• the records and data are
by being clear on how findings, criteria and conclusions were reliable and complete;
developed in a balanced and reasonable manner and why the
• the level of investment to
combinations of findings and criteria result in a certain overall develop a quality product is
conclusion or set of conclusions. If done properly, the intended users commensurate with risks; and
can be confident about the validity of the conclusions, and the auditor • clear responsibility and
has then provided assurance (GUID 3910/32). The assurance provided accountability for all levels are
established for managing quality
to the intended user has to be communicated transparently. throughout an engagement,
including engagement design,
staffing, stakeholder involvement,
As an auditor, you need to assess audit risk and take steps to provide message development and
assurance. Specifically, you need to: product review.
(1) identify the risks;

(2) assess those risks;
(3) develop and implement strategies to prevent and mitigate the risks; and
(4) monitor audit risk and mitigation strategies throughout the audit and make adjustments as
needed to changing circumstances.
Audit risk and the level of assurance are affected by numerous factors, but particularly important
is your audit team’s ability to:
• develop a quality audit design that comprises the scope of the audit and the appropriateness
of the evidence-gathering procedures;
51
• sufficiently understand the subject matter to actively manage audit risk and design. The audit
teams have to consider the conditions of the subject matter and the level of confidence
needed by the intended users of the audit report; and
• effectively exercise the use of professional judgement and professional scepticism in assessing
risks, as each audit topic is unique. You have to research and learn carefully about the topic
being audited and document your understanding of the subject matter in a way that provides
confidence that you have properly understood it.
More details about assessing and mitigating audit risks are provided in Chapters 4 and 5.
What does communication with audited entities, external stakeholders, media

and the public involve?
The Standard
The auditor shall plan for and maintain effective and proper communication of key aspects of the
audit with the audited entity and relevant stakeholders throughout the audit process.
The auditor shall take care to ensure that communication with stakeholders does not compromise the
independence and impartiality of the SAI.
The SAI shall clearly communicate the standards that were followed to conduct the performance
audit.
The auditor shall, as part of planning and/or conducting the audit, discuss the audit criteria with the
audited entity.
SAIs adopt audit standards, processes and methods that are objective and transparent, including
procedures for obtaining comments on audit findings and recommendations from the audited entity.
Source: ISSAI 3000/55, ISSAI 3000/59, ISSAI 3000/61, ISSAI 3000/49, and INTOSAI-P-20, Principle 3
Your audit team does not work alone in conducting a performance audit. You and your audit
team should maintain effective and proper communication with the audited entities to obtain
the necessary information to conduct your analysis and reach appropriate conclusions. An audit
may focus on one audited entity or several entities. Communication with all relevant entities
52
involved is important. In addition to consulting with stakeholders
Auditors can conduct
internal to your SAI, such as lawyers, methodologists and technical effective communication
experts, it may also be useful to enlist the help of those external to the with the audited entity through:
SAI. For example, SAIs may contract out work to subject matter experts • face-to-face meetings with
in trade organisations or research firms for assistance with the audit. audited entity officials;
However, it is important to maintain independence if seeking this type • teleconference meetings; and
of assistance and not allow the expert to inappropriately influence the
• written communication, such as
audit conclusions. Lastly, a strategy to outreach to the media and the letters and email.
public may need to be considered (especially for high visibility or
Audit teams need to meet with
controversial audits) for those SAIs who interact with the media about the audited entity or entities at
their work. key points during the audit,
including holding:
• an initial meeting to discuss the

Communicating with audited entities
audit objective, the audit
questions, scope and
methodology, information
It is important that the audited entities be kept engaged regarding all requirements and timeframes;
relevant matters about the audit. This is important for developing a
• working meetings to obtain
constructive working relationship and helping to ensure that the audit information and data from the
team can achieve the audit objective(s) and conduct a high-quality audited entity(ies) and update
the audited entity on the progress
audit. Communication can include obtaining information relevant to of the audit; and
the audit and providing management and those charged with
• a meeting near the end of the
governance with timely observations about potential findings and audit in which draft audit findings
conclusions. SAIs may provide general minimum requirements to their are discussed, and there is an
opportunity for the audited entity
auditors regarding communication with audited entities. For example, to provide comments and
SAI guidance could direct when auditors have to communicate with additional information.
audited entities and the type or level of detail to be discussed. Or, for In their communication with the
example, SAI policy and procedures may require that recommendations audited entity, it is important for
auditors to:
made to an audited agency may not be so prescriptive and detailed that
the SAI might be seen as consultants as opposed to independent and • be professional, respectful,
courteous and receptive; and
impartial auditors.
• ensure that they maintain their
independence and impartiality.
It is recommended that your audit team communicate with audited
entities at regular intervals throughout the audit. Specifically, your
team could:
• begin the communication process with the audited entities at the planning stage of the audit,
and continue throughout the audit. As audited entities may not have prior knowledge of
performance auditing, it is important to introduce the purpose and process of performance
auditing to them;
53
• engage the audited entities during the early stages of the audit when developing the: audit
subject matter; audit objective(s) and questions; audit criteria; the period to be audited; and
government undertakings, organisations and/or programmes to be included in the audit.
Access to documentation, data, the confidentiality/sensitivity of the information that will be
shared and how it can be used and disclosed in the final audit report are key matters to discuss
with the auditee(s) early in the audit process, preferably during audit planning;
• hold update meetings with the audited entities throughout the audit process and consider its
feedback. Audits often evolve as the audit team learns more about the topic and information
is obtained and analysed. You should keep the audited entities informed of any significant
changes to the key aspects of the audit, such as any changes to the audit questions or criteria.
Effective communication can help improve your access to information and data, help gain
better insights into the perspectives of the audited entities; and
• provide the audited entities with an opportunity to comment on the audit findings,
conclusions and recommendations. Additionally, the audited entities’ comments can be used
for correcting factual errors and considering the need to make other changes to the final
reports. The remaining differences of opinions or other important comments, along with the
SAI’s responses, may be published as part of the report.
A sound dialogue throughout the audit process that involves the audited entities is important in
achieving meaningful improvements in governance and may increase the impact of the audit. In
this context, you can maintain constructive interactions with audited entities by sharing
preliminary audit findings and perspectives as they are developed and assessed throughout the
audit. However, remember that you must also always maintain proper independence and
impartiality while effectively communicating and working with audited entities.
Communicating with external stakeholders
We discussed the importance of the SAI assembling audit teams that collectively have the
knowledge and skills necessary to conduct the audit and consult with stakeholders within the SAI,
such as experts or methodologists, through all audit phases. It is also appropriate to engage with
stakeholders external to the SAI. Potential stakeholders outside your organisation may include:
• academic and business communities;
• international bodies;
• internal auditors;
• citizens and their representatives;
• research institutions;
• civil society organisations (CSOs);
54
• professional institutions;
• representatives of vulnerable groups;
• other non-government organisations; and
• legal experts, if expertise does not exist within the SAI.
Stakeholder communication is important for both SAI leadership and audit teams. For example,
SAIs needs to cultivate good relations with various organisations to promote productive
collaboration.
In addition, you, the auditor, should strive to maintain good professional relationships with all
stakeholders involved in the audit. In doing so, promote a free and frank flow of information as
far as confidentiality requirements permit and conduct discussions in an atmosphere of mutual
respect and understanding of the respective role and responsibilities of each stakeholder. While
stakeholder communication is important, it is essential that this communication does not
compromise the independence and impartiality of the audit or the SAI. For example, your SAI
may have policies and procedures that dictate the types of details about the audit or audit
documentation that can be shared with stakeholders external to your SAI.
Communicating with the media and the public
A strategy for outreach to, and communication with, the media may be important to inform the
public of the outcome of audit work. It is good practice to make reports accessible to the public
and other interested stakeholders through the media unless prohibited by legislation or
regulations. Reporting audit results publicly, unless specifically limited, make the results less
susceptible to misunderstanding and facilitates follow-up to determine whether appropriate
corrective actions have been taken. It is important that SAIs make reasonable efforts to consider
the needs of the public and the media in their requests for information about the SAI’s work.
SAIs’ treatment of all media – whether print or electronic, local or national, domestic or
international – should be balanced and equitable.
As an auditor, it is important that you follow your SAI’s guidance or rules concerning
communicating with the media and the public. For example, SAI guidance might direct what level
of officials within the SAI can participate in media interviews.
55
The principles for conducting a quality performance audit span the entirety of your
work, so remember to always ...
… understand and act in accordance with your … determine the materiality of audit topics and
SAI’s quality control and assurance framework; findings, appropriately document the evidence
and decisions in the audit, and ensure effective
… consider independence, be aware of possible supervision of the audit;
threats to independence, and report them if
necessary; … assess audit risk and put in place strategies to
provide assurance in the audit;
… adhere to all ethical standards and
requirements of your SAI; … plan for and maintain effective and proper
communication of key aspects of the audit with
… exercise sound professional judgement and the audited entity and stakeholders; and
scepticism;
… keep in mind that performance audits require
… ensure your audit team collectively has the significant judgement, interpretation and
necessary professional competence to perform scepticism because evidence associated with
the audit; performance audits is typically persuasive rather
than conclusive, requiring constant reassessment.
56
IDI’s considerations to mainstream inclusiveness and maximise the impact of
performance audits
Besides the general principles coming from the performance audit ISSAIs, we would like to
highlight two cross-cutting considerations for performance audits: audit impact and
inclusiveness. These considerations are not performance audit requirements, i.e. the
performance audit can still be ISSAI complaint if these actions are not carried out. However, IDI
recommends that SAIs mainstream audit impact and inclusiveness considerations throughout the
performance audit process to enhance the value and benefit delivered by the SAI.
Impact driven performance audit process
IDI describes ‘audit impact’ as the contribution of SAI audit work to long-term positive effects on
people and the planet (a society/on a group/area), especially those left behind. Such audit impact
is achieved through a value chain of SAI outputs and SAI outcomes. Figure 7 is an illustration of
what such value chain could look like in case of performance audits.
Figure 7: Value chain of SAI outputs and SAI outcomes
SAI outputs SAI outcomes SAI contribution to

impact
•Adequate coverage •Legislative follow-up • Enhanced

•Executive action on governance and
•High quality audit
recommendations service delivery
reports issued in a
timely fashion positively impacting
people and planet
•Extensive outreach
of PA audit report to
diverse stakeholders
While SAIs have control over SAI outputs, there are many factors that affect SAI outcomes and
contribution to impact. SAIs are a part of an ecosystem. The social, economic, political context in
the country and multiple stakeholders such as legislative bodies, executive, civil society
organisations, professional bodies, academia, media etc., play a role in achieving audit impact.
Though a SAI may not have control over these, a SAI does have influence. To maximize the
possibility of SAI contribution to impact through performance audits, we recommend that the
57
SAI incorporate audit impact considerations throughout the audit process. This can be done by
asking and answering some key questions during different phases of the performance audit.
Key questions
• What is the envisaged audit impact of this performance audit?

• Will the topic selected contribute to desired audit impact?
• Will the audit design framework lead to desired audit impact?
• Are key stakeholders engaged throughout the audit process?
• Will the audit recommendations lead to a positive impact, including for those left behind?
• Are key messages conveyed to all relevant stakeholders?
• Are there processes in place for appropriate follow-up and facilitation of audit impact?
Some tips for enhancing audit impact are presented below.
Tips for enhancing audit impact
✓ Engage with SAI Leadership

✓ Communicate continuously with audited entities
✓ Create a stakeholder coalition
✓ Communicate the value of your work
✓ Reflect on audit impact throughout the audit process
✓ Follow ISSAIs
Mainstream inclusiveness considerations into performance audits

Millions of people across the world continue to live in poverty and are denied a life of dignity.
There are enormous disparities of opportunity, wealth and power. Gender inequality remains a
key challenge. The current pandemic has sharpened inequalities. People get left behind when
they lack the choices and opportunities to participate in and benefit from government processes
and outcomes. People can be vulnerable and left behind due to many factors like gender,
ethnicity, age, class, disability, sexual orientation, religion, nationality, indigenous, migratory
status, socio-economic status, geographical remoteness, conflict etc. In each national context, it
58
is important for those charged with governance to ensure that government policies, programmes
and institutions are inclusive and responsive to the needs of the marginalised.
Inclusion of the marginalised is important in both outcomes and decision-making processes of

government. As performance audit seeks to contribute to effective governance and service
delivery, it is important to examine if those charged with governance have been inclusive and
responsive to the needs of marginalised groups in their national context.
You can examine inclusiveness in performance audits by:
• Examining inclusiveness as a part of every performance audit topic. For example, in auditing
strong and resilient national public health systems, one of the topics examined could be the
preparedness of such systems to respond to the needs of marginalised sectors of the
population during an emergency. For example, people with disabilities, migrants and refugee
populations.
• Selecting topics that directly impact the marginalised. Based on national priorities, you can
decide to select high priority topics that directly impact the marginalised. For example, if you
are in a country with very high rates of violence against women, you could select the
elimination of violence against women as a performance audit topic.
• Engaging with stakeholders and beneficiaries from marginalised sectors. The audit process
itself can be inclusive by engaging with civil society organisations (CSO) that represent relevant
marginalised groups or reaching out to the marginalised sectors. Such engagement would
have many benefits, such as contributing to a better understanding of the subject matter,
ensuring that the voices of these sectors are heard and considered in all phases of the audit.
Such engagement would also be beneficial for the ability to formulate relevant audit
recommendations.
• Understanding the impact of the audit on marginalised sectors. Inclusiveness could also be
considered in understanding the impact of your audit. What difference will your audit make
to the marginalised sectors?
• Communicate key messages from your audits to create greater awareness of issues faced by
the marginalised sectors.
59
Questions you could ask to examine inclusiveness
✓ How are marginalised groups identified by the government and considered in the implementation of
policies?
✓ Who is being left behind, and what are the underlying reasons for their vulnerability?
✓ What disaggregated sources of data are available, and what are the data gaps?
✓ What actions are being taken to determine the needs of the marginalised?
✓ How does the government ensure that marginalised groups are included in decision-making processes?
✓ How does government ensure that marginalised groups are informed about government decisions and
actions?
✓ What action has the government taken on SAI recommendations related to marginalised and
vulnerable populations?
60
The Performance
Audit Process
Important steps and concepts
Step 1 Step 2 Step 3 Step 4 Step 5 Step 6

Selecting an Designing Conducting Developing Writing Following
audit topic the the audit findings, the report up on the
audit conclusions, audit results
and
recommendations
Selecting an audit topic

• Understand interests and priorities from the
Cross cutting
ministry, legislature, government, or other
considerations stakeholders such as civil society
organizations or the public.
• Use selection criteria to ensure audit topics
are significant, auditable, and consistent
with the SAI’s mandate.
• Scan the audit environment by conducting
risk, financial, and policy analyses.
• Prioritize audit topics and determine the
SAI’s highest priorities.
• Select a topic for the audit team.
61
Designing the audit
• Conduct a pre-study to better understand
the audit topic.
• Determine the audit approach.
Results oriented approach: are • Develop the objective(s) to establish the
outcomes being achieved? reason for the audit.
Problem oriented: what are the • Formulate audit questions to guide the
causes of the existing problems? specific areas of the audit.
System oriented: is the management • Identify suitable audit criteria to measure
system functioning properly?
the audited entity’s performance against
what is expected.
• Develop the methodology to guide the
collection and analysis of information.
• Document the design, such as with a matrix,
and develop a project schedule.
Materiality Conducting the audit

• Understand the importance of collecting
sufficient and appropriate evidence.
• Gather information and data by
employing the approved methodology.
• Analyse the collected information and
data using qualitative and quantitative
methods.
Developing findings,
conclusions and
recommendations
• Identify findings of the audit.
• Develop the message with appropriate
Condition balance on positive and negative
Criteria findings.
Cause • Draft conclusions and recommendations,
Effect if applicable.
Writing the report

• Establish a report structure that will
effectively communicate the audit
results.
• Draft the report in accordance with your
Communication SAI guidance.
• Obtain the audit entity’s comments on
the draft report.
• After receiving SAI management
approval, finalise and publish the report.
• Communicate the audit results to the
relevant parties.
62
Following up on
audit results
• Determine progress on the audit findings
and recommendations.
• Assess if the problems found were
addressed.
• Determine financial and non-financial
benefits.
• Identify areas for future audits.
The 3Es of auditing performance
Economy Efficiency Effectiveness

Keeping costs low Making the most Meeting objectives
of available and achieving
resources intended results
Tips
Use professional
Seek expertise Communicate judgement by Consider Consider balance
from stakeholders with the audited applying inclusiveness by reporting
internal to the entities knowledge, when designing, positive results
SAI as well as throughout experience, and conducting, and as well as
external experts. the audit. scepticism to each reporting. deficiencies.
audit phase.
63
Chapter 3
How do you select performance audit topics?
Selecting an audit topic

• Understand interests and priorities from the ministry, • Scan the audit environment by conducting
legislature, government, or other stakeholders such risk, financial, and policy analysis.
as civil society organisations or the public. • Prioritise audit topics and determine the
• Use selection criteria to ensure audit topics are SAI’s highest priorities.
significant, auditable, and consistent with the SAI’s • Select a topic for the audit team.
mandate.
According to ISSAI 3000, performance audit planning has two parts: selection of topics and
designing the audit. This chapter is about selection. It explains how Supreme Audit Institutions
(SAIs) can choose which topics to audit. Chapter 4 is about performance audit design.
The selection process may vary between SAIs. It is important for you, the auditor, to understand
how topic selection occurs as you conduct your work. This chapter offers general guidance based
on the requirements of performance audit International Standards of Supreme Audit Institutions
(ISSAI) and common SAI practices.

• What is the strategic planning process?
• How do SAIs scan the audit environment to identify possible topics for performance audits?
• How do external stakeholder requests arise?
64
• Why might a SAI consider auditing a topic that is not the responsibility of a single audited
entity?
• What criteria do SAIs use to select topics for performance audits?
What is the strategic planning process?
The Standard
The auditor shall select audit topics through the SAI’s strategic planning process by analysing potential
topics and conducting research to identify audit risks and problems.
Strategic planning is the process of determining the long-term goals of the SAI and identifying
the best possible approach to achieving them.11 The SAI´s strategic planning process may be
understood as the first step in topic selection because it comprises the analysis of potential areas
for audit and defines the basis for the efficient allocation of audit resources (ISSAI 3000/92). As
part of this process, the SAI researches to identify major risks and problem areas considered
important. The SAI analyses these subjects to identify which performance audits are of most
interest to the public, government and the legislature; and which ones can add the most value.
The strategic planning process used varies between SAIs. Plans normally cover several years and
guide SAIs in selecting topics for performance audit. The strategic plan will normally result in a
lower-level operational audit plan, indicating which topics will be addressed in the next one or
more years.
11
This is different from an operational or organisational strategic plan. The strategic audit plan will, however, inform
the organisational plan.
65
The objectives of a strategic audit plan are usually to …
… provide a firm basis for the SAI’s senior … communicate the SAI’s performance audit
management to set the direction for future audit priorities to audited entities and the legislature;
coverage;
… produce a work programme that can be
… understand the risks facing audited entities achieved with the available resources; and
and take these risks into account in audit topic
selection; … provide a basis for SAI accountability.
… identify and select performance audits with

the potential to improve public sector
accountability and administration;
How do SAIs gather information for the strategic planning process?
Audit topics derive from two main sources:

• General issues that the SAI identifies through scanning the audit environment.
• Requests or suggestions from stakeholders.
Figure 8 is an example that shows how these two sources can help the SAI select audit topics.
Figure 8: How SAIs could select audit topics
Input Input Output

Stakeholders’ suggestions:
• legislature; Understanding • Audit strategy plan.
• government/executives; stakeholders’ expectations • Potential audit topics.
• internal stakeholders; • Priorities.
• non-government organisations; • Resource requirements.
• other relevant stakeholders.
General issues: Topic selection matrix

• previous audit;
• government views, budget
papers, etc;
• agencies’ annual reports and
evaluation;
• media and external reports;
• previous audit fieldwork;
• analysis of performance
indicators;
• discussion with agencies/entity;
• priorities of legislature; Annual work plan
• priorities of government.
66
How do SAIs scan the audit environment to identify possible topics for
performance audits?
SAIs normally treat the identification of new performance audit topics as an ongoing process
rather than a discrete activity. Fruitful ideas can arise at any time and from many sources. Your
SAI and you, the auditor, have to remain alert for new challenges, risks and events that affect
government entities.
Audited entities face internal and external pressures that might make their work appropriate for
a performance audit. Examples of typical pressures include:
• budgetary constraints;
• economic, social and demographic trends;
• launch of new and complex programmes;
• availability of sufficiently-skilled staff;
• media focus on the entities’ activities; and
• changes in senior management.
It is important that SAIs evaluate changing and emerging risks in the audit environment and
respond to these in a timely manner (INTOSAI-P-12/Principle 5). During the strategic planning
process, techniques such as risk analysis or problem assessments can help structure the process.
However, they need to be complemented by professional judgement to reflect the SAI´s mandate
and to ensure that significant and auditable audit topics are selected (ISSAI 3000/93). If you
identify risk early and, through a performance audit, make recommendations to mitigate it, you
will hopefully have a significant impact on the effectiveness of your audited entities. Chapter 4
provides more details on how to identify and assess risks.
Many SAIs carry out an annual programme of updating risk assessments for the entities they
audit. This process helps them identify topics where they are likely to have a positive impact.
There are many different techniques for identifying possible audit topics. Figure 9 lists some of
the most common ways, while Figure 10 describes how the process operates at the SAI in Zambia.
67
Figure 9: Methods of identifying possible audit topics
Scanning the SAIs monitor key issues in the public sector to keep abreast of developments that
public sector might merit further scrutiny via a performance audit. For example, you might:
environment • read relevant publications and previous reports relating to performance,
financial and compliance audits;
• listen to the experience of other auditors;
• review transcripts of parliamentary debates;
• attend conferences and seminars;
• have discussions with colleagues, stakeholders and specialists; and
• consider media coverage of issues.
Area watching is a continuous process that ensures that you and the SAI are
always in possession of updated information about what is happening in society
and what areas may require further examination.
Reviewing SAIs look out for official announcements and publications that will affect their
official audited entities. The following list of information might be inspected by you:
• The international community’s sustainable development goals (SDGs).
announcements
• Resolutions by the country’s Committee on Public Accounts or comparable
committee.
• A speech from the Head of State that marks the opening of the parliamentary
year.
• Legislation and legislative speeches.
• National budgets and guidelines.
• Other public policy documents (for example, ministerial strategy papers, white
papers).
• Annual reports of audited entities.
• Global developments, such as themes identified by INTOSAI.
Financial Basic financial analysis includes being aware of how money flows into and out of
analysis the audited entity. You may choose to look more closely at material features such
as:
• complex financial arrangements;
• new sources of income and expense; and
• areas where spending is high or changing rapidly.
68
Media SAIs monitor the media (for example, newspaper articles, broadcast news and
monitoring social media) to identify concerns that the public or commentators are raising
about public services. It is important for you to consider a wide range of media so
that you can detect issues that may only affect certain segments of the
population. For example:
• Publications aimed at older people may provide clues to emerging issues in
areas such as pensions or treatment of health conditions that are more
prevalent among the elderly.
• Regional publications may draw more attention to the allocation of funding
from central government for activities such as public transport, sanitation and
telecommunications.
• Publications aimed at specific genders, ethnic groups or other similar segments
of the population may identify public service issues (such as health outcomes)
that are having a disproportionate impact on their readers.
A general overview (also known as a general survey) typically provides you with
General an understanding of an audited entity’s objectives, main activities, and the level
overviews and nature of resources used in carrying out its functions. You can assemble and
evaluate information on the background, objectives, activities, plans, resources,
procedures and controls in the entities or areas concerned.
The general overview aims to:
• identify and review those areas absorbing a significant level of resources;
• identify potential risks to achieving optimal use of resources;
• highlight areas for continuing audit attention; and
• propose areas or subjects for auditing.
You may carry out general overviews that cover:
• a whole audited entity;
• a group of related activities; and
• major projects or programmes of expenditure or receipts.
The general overview can be a valuable source of reference when making
proposals for inclusion in the strategic audit plan. When you gather information in
your general survey work, it is important to record all relevant facts and
assessments in easily-accessible working folders and keep them up to date.
A general overview typically covers:
• Background information on the audited entity.
• Significant legislative authority.
• Objectives of the audited entity.
• Organisational arrangement.
• Accountability relationships.
• Activities carried out.
• Nature and level of resources used.
• Procedures and control systems in place.
69
Consideration SAIs aim to be aware of how citizens view the performance of the audited entity
of views of and how their interests are affected by this performance.
citizens Citizens can be a source of ideas for performance auditing, a source of demand
for performance auditing and, at the same time, the users of performance audit
reports. SAIs aim to maintain relevant information outlining the views of the public
on the operation of government organisations or programmes.
In your analysis, you should consider inclusiveness – consider how the performance
of the audited entity affects different sectors of society. For example, when looking
at a programme to promote business skills, you could consider whether men and
women have similar access to the programme and how male and female
participants feel about their experience of the programme. If you identify material
differences in the experiences and outcomes for different social groups, you could
then investigate whether there are any changes that would make the services
more inclusive.
When considering whether a study topic is material, you might consider
stakeholder concerns, public interest, regulatory requirements, and consequences
for society.
Liaison with SAIs can build relationships with external stakeholders and interact with them
other frequently to identify and discuss possible audit topics.
external You may obtain input on audit topics from subject experts, relevant parties in
stakeholders government and the audited entity’s internal auditors.
The academic community contains people with expert knowledge in specific
audit areas. Such academics may provide a more objective view, less restricted
by personal interest. Academics may thus serve as suitable discussion partners and
sometimes also consultants at all stages of an audit.
Non-government organisations can be a useful source of ideas. They may have
conducted their own research through surveys and case studies and may have a
range of relevant contacts.
Internal You can engage with other performance, financial and compliance auditors
discussions and within your SAI to identify possible audit topics. For example, financial auditors
assessments may have found financial weaknesses that suggest that a programme has not
within the SAI been implemented effectively.
70
Figure 10: How SAI Zambia scans the environment to identify potential audit topics
SAI Zambia – using risk assessment in annual planning
SAI Zambia uses area watching as a primary annual plans, debates in Parliament, decisions
basis for conducting performance auditing. The made by the executive, allocations in budgets,
objective of area watching is to assess areas in the media, and public discussions by subject
the various sectors where there are reports of matter experts and other stakeholders.
dissatisfaction by the public with services or
goods provided by public entities or where The sector assessments are later compiled into
there are performance-related problems by an overall risk assessment, where selected
entities. They also use compliance and financial possible performance audit topics in different
audit reports to identify potential areas for sectors are presented and prioritised.
performance audit investigation.
Depending on the results of the assessments,
They allocate the responsibilities for area they determine whether the areas are
watching of certain sectors to groups of three auditable or not, considering the materiality,
auditors, at most. The task for the auditors is to availability of information, potential for change,
keep abreast of developments and problems in issues of public interest, etc.
the sector during the year. Area watching is a
continuous process where the auditors collect
information through, for example, strategic and
Source: AFROSAI-E Performance Audit Handbook, 2016
How do external stakeholder requests arise?

A ministry may ask the SAI to take an early look at how well a new programme works. SAIs may
also receive specific requests for investigations or audits from the legislature. In responding to
external stakeholders, the SAI is normally free to accept suggestions. However, while respecting
the laws enacted by the legislature that apply to them, the SAI must ensure that it retains its
independence. Unless specified otherwise by national law, the ultimate choice on whether to
conduct an audit and how to define the key research objectives of the audit should always lie
with the SAI, not the external stakeholder. (INTOSAI-P-10/Principle 3)
It is important for you, the auditor, to consider the needs and interests of your audit report’s
audience as you consider external requests. By taking these requirements into account, you can
ensure the audit report is useful and understandable. For example, you have to consider which
issues and findings are material to readers of the report. As noted earlier, materiality can be
defined as the relative importance or significance of a matter within the context in which it is
being considered. Besides monetary value, materiality includes social and political significance,
compliance, transparency, governance and accountability. (ISSAI 3000/84)
To deliver as much value as possible, the auditor shall consider materiality at all stages of the
audit process, including the financial, social and political aspects of the subject matter (ISSAI
71
3000/83). A matter can be judged material if knowledge of it would be likely to influence the
decisions of the intended users. Determining materiality is a matter of professional judgement
and depends on the auditor’s interpretation of the users’ needs. Your judgement may relate to
an individual item or a group of items taken together.
Why might a SAI consider auditing a topic that is not the responsibility of a single
audited entity?
The strategic audit plan may include topics that are not easily assigned to a single audited entity.
A performance audit may focus on a single programme, policy, entity or fund, or may focus on
outcomes or systems, looking across programmes, policies and entities that contribute to the
outcome or system. Following an assessment of the complexity of the subject matter and
mapping of responsible entities (including the relationship between them), the SAI has to decide
on the objective and scope of the audit. This activity may or may not happen already at the
strategic planning stage.
In some cases, there will be more than one entity responsible for the audit topic. For example, a
programme to use new technology to improve the productivity of agriculture might involve the
ministries that cover farming, the environment, finance, training and international trade and will
need them to coordinate to achieve the intended results.
The risk of performance problems is greater when different organisations with at least partly
different objectives contribute to the implementation of the same policy or activity. When you
consider the 3Es (economy, efficiency and effectiveness), this translates into a heightened risk of
underperformance. For instance, consider the case of a government that wants to enhance public
health by increasing the protein intake of the population. To do so, two ministries – health and
agriculture – will need to work together. It is quite possible that the Agriculture Ministry wants
to enhance earnings for farmers by selling meat products abroad, whereas the Health Ministry
wants the meat to be consumed domestically. It is easy to see how these tensions might
undermine the efficiency and effectiveness of the overall government policy.
When considering the audit topic across programmes, policies or entities, it is important to adjust
the audit process accordingly, from identifying the audited entities and their responsibilities to
establishing scope, criteria and methodology.
By taking care to identify these activities as viable audit topics, SAIs can:
• meet expectations that performance audits will cover all public bodies responsible for
spending public money and other public resources;
72
• promote closer collaboration between public bodies; and
• identify topics where no one is taking responsibility. To help find these gaps, a useful exercise
can be to map out the lines of responsibility that exist for a given activity or theme. You can
use RACI analysis to do this.12
SAIs will also want to consider whether their selection of individual topics fits in with any longer-
term strategy the SAI may have, such as providing balanced coverage across government. The
SAI may also aim to cover one large topic in several reports. For example, both the International
Organization of Supreme Audit Institutions (INTOSAI) and the United Nations have stressed, as
countries start to implement the SDGs, that SAIs can contribute to the success of the Sustainable
Development Agenda by auditing preparedness to implement the SDGs and tracking progress.
To do so effectively, SAIs might, for instance, produce a series of performance audit reports that
make timely recommendations.
What criteria do SAIs use to select topics for performance audits?
The Standard
The auditor shall select audit topics that are significant and auditable, and consistent with the SAI´s
mandate.
The auditor shall conduct the process of selecting audit topics with the aim of maximising the expected
impact of the audit while taking account of audit capacities.
Source: ISSAI 3000/90-91
Once the potential audit topics are identified, the SAI prioritises them to deploy its resources and
time efficiently and effectively. In selecting a performance audit topic, ISSAI 3000 states that the
SAI has to consider:
• the significance (including the financial, social and political aspects) of the subject matter;
• the auditability of the chosen topic;
• whether the SAI has the resources and skills to carry out the audit;
• whether the audit topic would be consistent with the SAI’s mandate; and
• how to maximise the impact (financial or otherwise) of the audit.
12ARACI analysis is a tool that identifies, for a set of activities, who is Responsible, who is Accountable, who has to be Consulted,
and who has to be Informed.
73
SAIs can develop their selection criteria and procedures in line with the requirements of the
ISSAIs. Two such methods – using a scoring matrix and comparing short summaries – are
described below.
Using a scoring matrix to select audit topics
A scoring matrix uses scores, supplemented naturally by professional judgement, as one indicator
of which audit topics might be chosen. The SAI chooses selection criteria, then scores each
potential audit topic against those criteria.
The criteria presented in Figure 11 are examples that may be considered in prioritising and
selecting the most viable audit topics. Please note that the criteria discussed here may not be
exhaustive or necessarily relevant to all SAIs. The relative importance of each criterion will
depend on the unique circumstances and context of each SAI.
74
Figure 11: Illustrative list of selection criteria for audit topics
Materiality Relative importance (or significance) of a matter in the context in which it is being
considered. In addition to monetary value, materiality includes issues of social and
political significance, compliance, transparency, governance and accountability.
Auditability Can the topic be audited? Is it practical to audit? Does it fall within the legal
mandate of the SAI? Does the SAI have the capability to audit the topic (for
example, does it have access to experts who understand the audit topic)?
Possible Will the topic have a powerful effect on enhancing the economy, efficiency
impact and/or effectiveness of government undertakings?
Risks to the Is there a strategic or reputational risk if an audit topic was not examined?
SAI
Legislative or Will auditing the topic address a legal concern or be to the advantage of the
public interest community? For example, will the audit help to promote inclusiveness?
Relevance Does the topic have some bearing on, or importance for, real-world issues,
present-day events or the current state of society?
Timeliness Is this the right or appropriate time to audit the topic? For example, is it too early to
examine progress of a new activity?
Previous audit Has the topic been audited in the past? Is it worth auditing it again? Is there a new
work audit approach you could take?
Other major Is other work being planned or done on the topic?

work planned
or in progress
Request for Have any special requests been made for performance audits to be done?
performance Consideration should be given to the source of a request to determine its
importance, for example, requests from parliament, beneficiaries or other external
audits
stakeholders.
Based on the criteria discussed above, the potential audit topics can be ranked and prioritised. It
is important to highlight the need for your professional judgement in the selection process.
Appendix 4 provides an illustration of audit topic selection using an audit topic selection matrix.
The mathematical score presented there is a tool that can help to identify important topics to
audit, but it is not a substitute for your professional judgement.
75
Comparing short summaries of each possible audit topic
This approach does not use a scoring system. Instead, audit teams prepare simple, short
summaries of the possible audit topics using a standard template. Senior management can then
easily review each option to see which ones fit with their strategic priorities. Figure 12 illustrates
how a summary can be used.
76
Figure 12: Sample summary of an audit topic
Title Waste management
Context Solid waste management is a vital quality-of-life and health issue for citizens. The
government is spending an increasing amount on it, often dealing with private
sector providers.
Rationale SAI has not looked at the topic in the last eight years. The government has recently
awarded a large contract for the next five years to a company that has
performed poorly in other public service contracts and is in a weak financial
position.
What the audit The audit would look at whether the ministry is getting good value from its
could achieve contracts with private sector suppliers, with important lessons for the future. For
example, we would examine the efficiency of the procurement process. Early
intervention might also lead to improving health outcomes for citizens.
Key risks • Our audit may be seen as coming too early in the life of the new contract.
• Our in-house expert on commercial contracting is shortly due to go on a one-
year secondment, so will not be available to assist with the audit.
• We already have two audits planned at the ministry, so they may feel that the
audit burden is too high.
• We will need to find a convincing international comparator against which we
can benchmark performance.
• When assessing performance, it may be challenging to estimate what would be
a ‘fair’ price for the services, as the ministry has never provided the services in-
house. As mentioned above, we would need to find a suitable international
comparator.
• When estimating the cost to public health of service failure, we will need to
provide strong evidence of a direct causal link between poor waste
management and national disease levels. We need to be very alert for other
factors that may contribute to increased incidence of disease.
Public Accounts Parliament is very interested in the topic. Several members of parliament have
Committee/ mentioned in recent debates that the current system is poor and they frequently
Parliamentary get complaints from their constituents about poor customer service and failure to
interest carry out vital repairs.
Type of output Performance audit
Indicative $300,000
budget
Indicative Audit will take nine months, reporting by September 2020.

timing
77
When selecting performance audit topics, remember to …
… select audit topics through the SAI strategic … use a wide source of information to scan the
planning process by analysing potential topics environment to select the most appropriate
and conducting research; audit topics; and
… ensure that audit topics are significant, … use appropriate criteria to help you evaluate
auditable and consistent with the SAI’s and select audit topics.
mandate;
… select audit topics that would maximise

impact while taking into account the SAI’s audit
capacities;
78
Chapter 4
How do you design a performance audit?
Designing the audit

• Conduct a pre-study to better understand • Identify suitable audit criteria to measure the
the audit topic. audited entity’s performance against what is
• Develop the objective(s) to establish the expected.
reason for the audit. • Develop the methodology to guide the
• Determine the audit approach. collection and analysis of information.
• Formulate audit questions to guide the • Document the design, such as with a matrix,
specific areas of the audit. and develop a project schedule.
• risk, financial, and policy analyses.
After selecting the audit topic, the second step of the audit planning is the audit design. Audit
design is a key step in completing a performance audit and is a critical component of
implementing an SAI’s quality control framework. It is also one of the most important aspects of
a performance audit, as the design will help ensure you obtain the knowledge you need to
complete your audit work.
The Standard
The auditor shall plan the audit in a manner that contributes to a high-quality audit that will be carried
out in an economical, efficient, effective and timely manner, and in accordance with the principles of
good project management.
Effective design consists of establishing a strategy for completing the audit and writing a detailed
audit plan that includes the audit type, timeline, resource requirements (people and money), an
overview of audit topic, scope (and limitations), objective(s), questions, criteria, risks, and
79
methodology. Developing a good audit plan is critical to laying the foundation for assessing
economy, efficiency and effectiveness in a performance audit.
As described in this chapter, audit design includes many sequenced steps; however, aspects of it
have to be revisited throughout the audit in response to changing information, resources and
timelines.
• How do you conduct a pre-study of the audit topic?
• How do you determine the approach for a performance audit?
• How do you develop audit objective(s)?
• How do you formulate audit questions?
• How do you determine the scope of the audit?
• How do you select audit criteria?
• How do you develop the audit methodology?
• How do you manage risk during audit design?
• How do you determine the time frames and resources for a performance audit?
• How do you document the audit plan?
• How do you involve internal and external stakeholders and management when designing a
performance audit?
How do you conduct a pre-study of the audit topic?
The Standard
To ensure the audit is properly planned, the auditor needs to acquire sufficient knowledge of the
audited program or audited entity’s business before the audit is launched. Therefore, before starting
the audit, it is generally necessary to conduct research work for building knowledge, testing various
audit designs and checking whether the necessary data are available. This preliminary work can be
called pre-study.
As an auditor, you need to take steps to ensure your audit is properly designed. To do this, you
will need to gather information on the audit topic and the audited entities´ business. You can
start by conducting preliminary work to build knowledge, think about possible audit designs and
assess whether the topic is auditable. Although your SAI already considered whether the topic
was auditable when selecting audit topics (as discussed in Chapter 3), circumstances could have
since changed, or you may reach a different conclusion after you conduct your preliminary work.
This preliminary work can be called pre-study. During the pre-study step, you will try to establish
whether conditions for a successful audit exist.
80
Specifically, as assessed during the selection of the audit topics phase,
An internal control is a
process that helps an you will need to determine whether the audit is still expected to add
entity achieve its
value to your SAI’s strategic objectives; enhance the audit topic´s
objectives related to things such
as running its operations economy, efficiency and effectiveness by strengthening internal
efficiently and effectively,
reporting reliable information
controls; and uncover fraud, waste and abuse. It is also important to
about its operations and develop an understanding of what is not working well – the performance
complying with applicable laws
and regulations.
weaknesses or problems that the audit may address. SAIs approach pre-
study differently. Some consider it to be a full-scale study conducted
prior to designing the audit, while others consider it to be a part of the
design phase of the audit. You need to conform to your SAI’s approach when completing this
step.
It is important to develop a sound understanding of the audit topic, as well as its context and
possible impacts, to facilitate the identification of significant audit issues and to fulfil assigned
audit responsibilities. Performance audit is a learning process involving the adaptation of
methodology as part of the audit itself. (ISSAI 3000/100).
This pre-study has to be done in a manner that conforms to your SAI´s processes and be
appropriately documented.
To determine whether conditions for a successful audit exist, you will need to build on work
completed when you selected your audit topic; that is, by collecting additional information that
enables you to understand:
• the organisational structures, roles and functions, stakeholders, activities and processes,
resources and trends;
• the organisational goals;
• applicable internal controls;
• the internal and external environmental factors that affect the entities and programmes under
review;
• the external constraints affecting the delivery of outputs and outcomes;
• what is working well and not working well within the entities and programmes under review;
• the criteria that exist or can be developed to assess performance; and
• the extent to which the activities are inclusive of all affected parties.
You will need to collect this information throughout the audit process; however, most of this
basic information has to be collected early in the audit during the design and conducting audit
work phases. Keep in mind that you and your audit team will need to be flexible and pragmatic
81
in the collection methods you use to obtain this information. In conducting the pre-study, you
will likely need to collect information from various sources, including those identified in Figure
13.
Figure 13: Information sources
Legislation, legislative speeches, ministerial Discussions with audited entity management

statements and government decisions. and staff and key stakeholders.
Strategic and corporate plans, mission Organisation charts, internal guidelines, and
statements and annual reports. operating manuals.
Interviews with experts, including non-governmental. Previous audit reports.
Policies, directives and plans. Reviews, evaluations and studies.
Performance and accountability reports. Media coverage.
Management information systems. Websites.
During the pre-study, it is also critical to ensure your team has documented its independence and
begins to work directly with stakeholders inside and outside your SAI, as appropriate. To do so
effectively, you may need to complete stakeholder analyses so that you can identify internal
stakeholders (for example, legal experts, methodologists and technical experts), their role and
interests, the anticipated degree of their involvement in the audit, and how important they are
in terms of the information they can provide. You may also need to do this for external
stakeholders, such as trade groups, associations and experts outside your organisation. For an
example of these analyses, see Appendix 5.
After identifying internal stakeholders, some SAIs bring them together

Involving internal
stakeholders like legal to participate in an initial meeting. During this meeting, you will discuss
experts, economists or with your stakeholders the possible approaches you will use, the audit
individuals with technical
expertise early in an audit can objective(s), audit questions, design options, and potential points of
enable you to quickly identify contact who know about the audit topic. If you hold this meeting, you
information and data sources,
points of contact, and might consider whether to provide documents to stakeholders in
methodologies that can help
advance to help facilitate the meeting discussion. For example, if the
you in planning your audit. It is
critical that you document key audit is complex or involves new issues or subject areas, the team may
decisions that your team
decide that it would be useful to provide additional materials or
reaches when meeting with
these stakeholders and that you background information to aid the meeting’s participants. By holding
maintain them in your audit files.
this meeting during pre-study, you may find that stakeholders are
better able to contribute to the development of your audit’s scope and
82
methodology. It is critical that you document the key decisions your team reaches due to the
meeting and maintain them in your audit files.
It is important that audit teams meet the audited entities before starting
Initial meetings with the
to collect information or data. During the initial meeting with the
audited entity can
audited entities, your team will discuss the reason for your work, enable you to leverage the
introduce your team to officials, provide your information needs for the expertise of officials to quickly
identify relevant information
audit, and discuss offices and site locations where you anticipate and refine the objective(s),
conducting your work, among other things. Meeting with officials from scope and methodology for
your audit. It is critical that you
the audited entities during the pre-study enhances your ability to obtain document key decisions your
team reaches as a result of the
the information you need to determine whether the topic is auditable meeting and maintain them in
and whether conditions exist for a successful audit. These meetings also your audit files.
enable your audit team to hear and take account of officials’ Source: IDI/PAS Development Team
perspectives and input early in the audit. As with initiation meetings, it
is important that you document the results of this meeting and any key decisions your team
reaches during the discussion. See Appendix 5 for a sample agenda used to guide this type of
meeting.
83
Steps that you can take when conducting the pre-study
✓ Review previous work on the topic by your ✓ Review performance and accountability
Supreme Audit Institution and conducting reports on the programme prepared by the
background research by reviewing responsible agencies. This can give the audit
documentation produced by the audited entity, team a better understanding of functional
including policies and performance reports. and financial details of these agencies and
reveal areas of primary concern.
✓ Consult with advisors and outside organisations
that have experience with the topic of the audit. ✓ Hold ‘initiation meetings’ with key internal
Organisations could include the United Nations, stakeholders to discuss possible approaches,
donor organisations, civil society organisations, research questions, design options and
non-government organisations and others with potential points of contact with knowledge
specialised expertise. of the audit topic.
✓ Analyse media reports on issues relevant to the ✓ Hold a meeting with the audited entity to
audit topic. discuss your information needs and enhance
your understanding of whether the topic is
✓ Review relevant government initiatives to auditable or whether conditions exist for a
understand their goals, approaches and funding. successful audit.
How do you determine the approach for a performance audit?
The Standard
The auditor shall choose a result-, problem- or system-oriented audit approach, or a combination thereof.
The overall audit approach is a central element of any audit, and it is an important link between
the audit objective(s) and the audit questions. There are three common approaches to
conducting a performance audit: a result-, problem-, or system-oriented approach. It is important
that you consider whether you anticipate using one or a combination of approaches when
developing your audit objective(s) and audit questions.
A result-oriented
A result-oriented audit approach assesses whether an outcome or output
approach seeks answers objectives have been achieved or services are operating as designed. In this
to questions like:
approach, you will express the findings in the form of a deviation from your
• What results have been performance criteria. Your recommendations will aim to eliminate these
achieved?
deviations by addressing their cause(s). In the result-oriented approach, you
• Have the audited entities will study performance in the economy, efficiency and effectiveness, and
met their objectives?
relate your observations to the goals, objectives, regulations or audit criteria.
If the criteria are difficult to determine, you may need to work with experts to
develop credible criteria.
84
A problem-oriented audit approach generally begins with a preliminary
A problem-oriented
problem that may or may not need to be further verified during the audit. approach seeks answers
to questions like:
Accordingly, this approach places a special emphasis on examining,
verifying and analysing the causes of performance problems. You can use • What are the causes of the
problem?
this approach when there is a clear consensus on a problem, even if there
is no clear statement of the desired outcomes or outputs. If you use this • To what extent can the
government solve the problem?
approach, your conclusions and recommendations will be based on the
process of analysing and confirming causes using criteria that allow you Source: IDI/PAS Development Team
to assess how specific factors contribute to the identified problem. A
major task in the problem-oriented audit approach is to analyse the causes of the problem from
different perspectives.
A system-oriented audit approach examines the proper functioning of

A system-oriented
approach seeks answers management systems. If you use this approach, you may find that
to questions like: performance benchmarks and principles of good management will be
• What are individuals’ roles and helpful as criteria in assessing the conditions for the economy, efficiency or
responsibilities? effectiveness, even when there is a lack of a clear consensus on a problem
• What are relevant information
flows? or when outcomes or outputs are not clearly stated. To help the users of
• Is there a good-quality reports understand the significance of weaknesses on the performance of
monitoring system?
• Are processes evaluated management systems, it is important to identify reasons for the
periodically and properly? weaknesses and establish plausible links to how weaknesses affect
Source: IDI/PAS Development Team operations. Well used, this type of audit seeks to answer a wide variety of
questions to describe how the activities are functioning, the cause of any
weaknesses, and the extent to which things could be improved.
As stated in ISSAI 3000/40, it is also possible to combine audit approaches. For example, an audit
of the implementation of SDG targets would be a combination of result and system-oriented
audit approaches. Such audit will use the Whole of Government (WoG) approach.
The WoG shifts the focus of government performance towards the results that government seeks
to achieve to address a societal problem or challenge rather than the operations of any single
programme, agency or entity. In this case, you will have a situation shown in Figure 14.13
13You can find more information on WoG at ISAM (https://www.idi.no/en/isam) and

https://www.effectiveservices.org/assets/CES_Whole_of_Government_Approaches.pdf
85
Figure 14: Relations when auditing more than one entity
Source: Some considerations on external audits of SDG implementation (Le Blanc, David and Montero, Aránzazu Guillán, 2020)
How do you develop the audit objective(s)?
The Standard
The auditor shall set a clearly defined audit objective(s) that relates to the principles of economy,
efficiency and/or effectiveness.
Audit objective(s) establish the reason(s) for conducting the audit. The objective(s) provide the
starting point for developing the audit questions that will guide your work. As a result, the
wording of the objective(s) is important and can influence the audit results. The audit objective(s)
should be designed to maximise the benefits and impacts from the audit, incorporate the concept
of materiality, and seek to evaluate economy, efficiency and effectiveness of the audit topic (see
GUID 3910/35-42 and GUID 3920/24-30). In setting the audit objective(s), you need to consider
the mandate of your SAI and the reasons for the audit.
You can think of the audit objective(s) as a neutral statement of the goal(s) for the audit. It
provides the basis for developing audit questions (discussed later in this chapter). Depending on
the needs of your audit, you have the flexibility to state your objective(s) as a phrase or to write
86
them as questions. Either way, you need to consider the following factors when developing your
objectives:
• Are the audit objectives framed in clear and simple terms?
• Are the objectives specific, feasible, fair and objective, policy-neutral and measurable?
• Are the objectives framed in a way that allows you to come to an unambiguous conclusion?
• Do the objectives provide sufficient information to audited entities and stakeholders to easily
understand why you are conducting the audit, the audit’s focus and the audit’s goal?
87
Figure 15: Examples of appropriately and inappropriately-formulated audit objectives by
audit approach
Audit approach Examples
Problem-oriented Protecting fish habitat

approach • Appropriate: Determine why the fisheries and environment department did not enforce key
fisheries statutes and habitat policy.
• Inappropriate: Determine how the fisheries and environment department has failed.
This objective does not provide the audited entity and stakeholders with sufficient information
to understand why you are conducting the audit, the audit’s focus, and the audit’s goal.
K-12 education: Public high schools with more students in poverty and smaller
schools provide fewer academic offerings to prepare for college
• Appropriate: Examine why public high schools with more students in poverty and smaller
schools provide fewer academic courses.
• Inappropriate: Examine why schools in impoverished areas are providing vocational training
instead of preparing students for college.
This objective implies that vocational training is a poor solution to the problem of educating
poor students and does not provide the basis for coming to clear and unambiguous
conclusions.
Results-oriented Assessment of officers in the entertainment sector

approach • Appropriate: Assess the extent to which officers have implemented key income tax provisions.
• Inappropriate: Assess entertainment sector compliance with the provisions of income tax rules.
This objective does not clearly identify who is being audited or the basis for the audit. It is also
broadly scoped and does not position the audit to arrive at clear and unambiguous
conclusions. It is more of a compliance audit than a performance audit objective. Finally, this
objective does not enable the auditor to distinguish between societal problems and problems
with government performance.
Farmers’ income stabilisation: Comprehensive set of tools, but low uptake of

instruments and overcompensation need to be tackled
• Appropriate: Assess the extent that risk and crisis management tools have been implemented
and are delivering intended results.
• Inappropriate: Assess which risk and crisis management tools have been best able to deliver
results.
This objective seeks to complete a result-oriented analysis without adequate criteria to quantify
‘best deliver results’. Scope is overly broad and objective may not be achievable.
Systems-oriented Assessment of use of government grants for education and monitoring of grant
approach recipient activities
• Appropriate: Assess the extent that agency systems include controls needed to monitor how
grant recipients use funds.
• Inappropriate: Assess whether agencies are monitoring grant recipients to ensure that funds
are being used appropriately.
This objective does not clearly establish the scope of the review and introduces subjectivity in
the terminology it uses that may be difficult to support using objective criteria.
Assessment of management system response to the tsunami disaster

• Appropriate: Assess the extent to which management systems and procedures permit a
sufficiently rapid and appropriate response to the tsunami disaster.
• Inappropriate: Assess whether additional steps could have been taken to respond to the
tsunami disaster more effectively.
This objective presupposes that there was a poor response and does not specify the target of
the audit, making it too broadly scoped to be actionable.
Source: Adapted from the Office of the Auditor General of Canada; Comptroller and Auditor General of India; and US GAO
88
How do you formulate audit questions?
After developing your audit objective(s) and approach, you will formulate specific audit questions
to guide your audit work. Audit questions should flow from the overall audit objective(s) and
typically are more specific to address the topics you will describe or evaluate during the audit.
The aim is for your audit questions to cover all aspects of the audit objective(s). Each of the
approaches described above may lead you to formulate your audit questions differently. As was
the case when developing your audit objective(s), it is critical that your team thinks carefully
about the wording of the audit questions because it will have implications for your decisions, the
types of information you will collect, your information and data collection methods, your
analytical approach, and the types of findings and conclusions you will reach. If you choose to
decompose your audit questions into sub-questions, ensure they are complementary, not
overlapping, and collectively exhaustive in addressing the overall audit question (GUID 3000/37).
Audit questions are either descriptive (meaning they describe a condition) or evaluative (meaning
they evaluate a condition against criteria and can be normative or analytical) (GUID 3920/31-37).
Descriptive audit questions can take multiple forms. Some are easily answered, while others are
more difficult. For example:
• What are the characteristics of recipients of the rural school programme?

• What is known about the number of workers involved in activity X, both those employed
directly by the government and those employed by companies contracted with the
government?
• What are the institutional arrangements put in place by the government to achieve vertical
and horizontal coherence of the activities related to sustainable public procurement?
• How is the government engaging with non-government stakeholders to implement initiatives
related to the elimination of violence against women?
Evaluative audit questions can vary widely, ranging from assessing a programme’s current
economy to prospective analysis of future events. There are five types of evaluative audit
questions, as shown in Figure 16. The use of such audit questions is not mutually exclusive. For
example, a performance audit with a question to evaluate effectiveness may also include
evaluating internal controls.
89
Figure 16: Types of evaluative audit questions by audit approach
Type of question Description Audit approach and example

1 Programme Questions that focus on economy Problem-oriented approach:
economy and and efficiency address the costs and • What factors explain the variation in costs
efficiency resources used to achieve of patient care among public hospitals?
programme results.
2 Prospective Prospective analysis questions provide Problem-oriented approach:

analysis analysis or conclusions about • What challenges, if any, will students in high-
information that is based on poverty schools face when preparing to
assumptions about events that may attend college in the next 20 years?
occur, along with possible actions that Results-oriented approach:
the audited entity may take in response • How might proposed federal standards for
to future events that may affect youth camp safety affect overall rates of
economy, efficiency or effectiveness. child injury and illness?
3 Programme Questions that focus on programme Results-oriented approach:

effectiveness effectiveness and results typically • To what extent do international school food
and results measure the manner and extent to aid programmes follow good practices for
which a programme is achieving its these programmes established by the
goals and objectives, and thus may also United Nations?
examine the quality of programme Systems-oriented approach:
implementation. • To what extent have management systems
and procedures permitted a sufficiently
rapid and appropriate response to the
disasters?
4 Internal controls Internal control questions relate to an Results-oriented approach:

assessment of an organisation’s system • To what extent do established internal
of internal control that are designed to controls help ensure the achievement of
provide reasonable assurance of desired results?
achieving effective and efficient Systems-oriented approach:
operations and reliable financial and • To what extent does the Ministry of Labour
performance reporting. ensure that performance measures for its
employment training programmes are
valid, reliable and complete?
5 Compliance Compliance questions relate to Systems-oriented approach:

compliance with criteria established by • To what extent have projects funded under
laws, regulations, contract provisions, the Highway Emergency Relief Program
grant agreements and other complied with federal programme
requirements that could affect the eligibility requirements?
acquisition, protection, use and
disposition of the entity's resources and
the quantity, quality, timeliness and cost
of services the entity produces and
delivers. As discussed, a performance
audit may include elements of a
compliance audit.
Source: Office of the Auditor General of Canada; Comptroller and Auditor General of India; and US GAO
There are several techniques you can use to craft audit questions. One way is to prepare an issue
analysis pyramid, such as the one modelled in Figure 17. The purpose of this tool is to break the
audit objective into a number of more detailed questions to form a pyramid. This allows you to
consider all dimensions of your audit questions. The audit objective, shown at Level 1, seeks to
90
evaluate the extent to which the health department has identified current and future costs of
providing prenatal care to impoverished populations. Level 2 identifies the audit questions the
team will need to answer during the audit to address the audit objective.
Figure 171: Example of issue analysis
Level 1 To what extent has the health department determined the

audit objective current and future cost of providing prenatal care to
impoverished populations?
Level 2 How much money did the health To what extent has the health
audit questions department expend in providing department estimated the future cost
these services during fiscal years to the government of providing these
2018 and 2019? services over the next ten fiscal years?
This technique can enable you to assess the feasibility of answering the audit question(s) and
develop the logic underlying your audit activity.
A second technique is to complete a cause-effect problem analysis, such as the one modelled in
Figure 18. Depending on your audit, this may entail completing two discrete steps. The first step
is to determine whether the expected results have been achieved or if a system is operating as
expected. If this is not the case, there may be a performance problem, and you would need to
consider whether the analysis could be brought a step further to hypothesize and analyse the
causes. In the problem-oriented approach, the main performance problem and preliminary
identified main factors causing it can be part of the original audit design.
The effect, shown at Level 1, provides the starting point for evaluating hypothesized causes
identified at Level 2. Potential causes provide the basis for developing audit questions, which are
identified at Level 3.
91
Figure 18: Example of cause-effect problem analysis
Level 1 Large numbers of people are injured or die in car accidents

effect
Level 2 Operator error is the Medical system is Poor infrastructure is

potential cause primary cause of limited in its ability to the primary cause of
injuries and fatalities treat injuries injuries and fatalities
Level 3 What steps are How is the What is the What is the
audit questions being taken to government capacity of condition of the
teach enforcing the medical roads and
individuals traffic safety system to treat infrastructure
how to drive laws? victims of car where
and maintain accidents? accidents are
their vehicles? commonly
occurring?
The formulation of audit questions is an iterative process in which you repeatedly specify and
refine the questions, taking into account known and new information of the subject and the
feasibility of obtaining answers. It is important that you allow your audit to evolve to obtain
additional information and further insights into sub-problems and causes. In doing so, it is also
critical that you document when, how and why you modify your audit questions so as to provide
a complete record of your audit. If significant changes are made, it is necessary to inform the
audited entities about the changes.
How do you determine the scope of the audit?

The scope defines the boundary of your audit and addresses such things as specific questions you
intend to ask and the type of study you will complete. In particular, the audit scope defines the
subject matter the auditor will assess and report on, the documents or records to be examined,
the period reviewed, and the locations that will be included. The scope is directly impacted by
the audit’s objective(s) and questions. As a result, you may need to modify the scope as you
collect information and become more knowledgeable about the subject of the audit. You will also
need to consider the impact that any changes in your audit objective(s) or questions may have
92
on the scope of your audit. Developing the scope of your audit is a critical part of audit design.
See GUID 3910/24-26 and GUID 3920/21-23 for additional information.
You can establish the scope of your audit by answering the questions listed in Figure 19.
Figure 19: Scope questions

What? • What specific questions or hypotheses are being examined?
• What are the key processes relevant to your audit?
• What is the subject matter that will be assessed and reported on?
• What resources are available to complete the audit?
• What questions, processes, and resources will not be covered?
Who? • Which agencies and organisations have responsibilities or perspectives relevant to the audit?
• Who within relevant agencies and organisations is best positioned to provide appropriate and
sufficient evidence to answer the audit questions?
• Who is responsible for assuring the reliability of information and data that are relevant to your audit?
• Which organisations or persons will be excluded?
Where? • What are the locations to be covered?

• Where are the documents and records that need to be examined?
• What locations will be excluded?
When? • What is the timeframe to be covered?
You will also need to consider many additional factors when deciding on the scope of your audit.
For example, you may need to limit your scope based on the availability of reliable sources and
data. You may also need to refine your scope based on:
• the resources available to execute the audit, including access to auditors with the skill sets
needed to implement complex methodologies, such as methodology experts;
• access to subject matter experts;
• the costs associated with travel; and
• the time constraints of the audit.
The scope of your audit may include any issues that led to recommendations in prior audit reports
if those issues remain relevant. The examples of scope below are adapted from various published
performance audit reports.
93
Examples of audit scope
1. Protecting Fish Habitat 2. Managing the expansion of 3. Management of Consumer

from 2009 Spring Report of the Academies Programme. Complaints. Report of the
the Commissioner of the Report of the National Audit Office of the Auditor General
Environment and Sustainable Office UK, 2012. of Botswana, 2008.
Development. Report of the
Office of the Auditor General The audit evaluated the The audit focused on the
of Canada, 2009. Education Department’s activities undertaken by the
implementation of the Consumer Protection Office
The audit included the programme expansion since to manage consumer
Administration of Fish Habitat May 2010 and the adequacy complaints during the
Protection, the pollution of its funding and oversight financial years 2003-2006.
prevention provisions of the framework across the The audit coverage was
Fisheries Act, and the two academic sector (including nationwide. Consumer
policies (the Habitat Policy academies established before complaints data for 2003-04
and the Compliance and May 2010). The expansion was financial years were not
Enforcement Policy) that set still in an early phase, and available.
out the government’s there was limited trend data
intentions relating to these on how schools had
provisions. The audit included performed academically
the policies, programmes since joining the expanded
and activities of fisheries and programme. The audit
oceans programme and examined this aspect of
certain arrangements with academies’ performance as
others that support the part of the future value-for-
administration and money programme. The report
enforcement of these did not cover capital funding
provisions. The audit did not nor assess in depth the impact
focus on the environmental of the expansion of local
assessments required by the authority finances or services.
Environment Assessment Act
that may be triggered by
ministerial authorisations
under the provisions of the
Fisheries Act.
How do you select audit criteria?
The Standard
The auditor shall establish suitable audit criteria, which correspond to the audit objective(s) and
audit questions and are related to the principles of economy, efficiency and/or effectiveness.
Once you have determined your scope, it is time to consider the criteria that will allow you to
measure the audited entities’ performance against what is expected.
94
What are audit criteria?
Audit criteria identify the required or desired state or expectation with respect to an audit
topic, representing reasonable and attainable standards of performance against which you
can assess the economy, efficiency and effectiveness of activities. In short, they are the
standards against which your audit evidence should be judged. In this sense, criteria provide
a context for evaluating evidence and understanding the findings, conclusions and
recommendations of an audit report (see GUID 3910/55-60 and GUID 3920/38-43).
Criteria are needed in all audits where performance is being evaluated. As stated, such
evaluations may include aspects of compliance when it is relevant to the performance of the
audited entities. Audit criteria can represent an expectation of ‘what should be’ according to
laws or regulations, ‘what is expected’ according to best practice or ‘what could be’ given
better conditions. Accordingly, criteria can be qualitative or quantitative, general or specific,
or a normative model (that is, norms related to aspects of compliance, when relevant to
performance, or economy/efficiency) for the subject matter under review. Examples of
criteria include:
• laws and regulations applicable to the operation of the audited entities;

• goals, policies and procedures established by the audited entities;
• technically-developed standards or norms;
• expert opinions;
• procedures for a function or activity;
• defined business practices;
• contracts or grant agreements;
• benchmarks or performance indicators set by the SAI, the audited entities or other
relevant entities or sectors;
• prior periods’ performance; and
• criteria used in similar audits or by other SAIs. (Note: You will need to ensure these criteria
are still valid.)
How do you choose audit criteria?
When selecting performance audit criteria, it is important to do so objectively. The process

requires rational consideration and sound professional judgement. Sometimes audit criteria
are easy to define, such as when the goals set by the legislature or government are clear,
precise and relevant. However, this is not always the case. For example, relevant criteria may
not be apparent at the outset of the audit, and applicable performance goals may be vague,
95
conflicting or non-existent. Similarly, you may find the criteria or standards set by the audited
entities do not equal good performance, requiring you to select, adapt or even develop
additional criteria that can provide more appropriate benchmarks of performance. In many
cases, you may find that a mixture of criteria from the audited entities and other sources
provides the right framework for assessing performance. However, in all such instances, it
will help if you perform some audit work before selecting your criteria in order to ensure
materiality and to enable you to become more knowledgeable about the issues and
associated best practices.
To objectively select audit criteria, it is important to have:
• a general understanding of the area to be audited and familiarity with relevant legal and
other documents, as well as recent studies and audits in the area to be audited;
• good knowledge of the motives and the legal basis of the audit topic and the goals set by
the legislature or the government; and
• a general knowledge of practices and experience in other relevant or similar government
programmes or activities.
It is essential that the criteria you select are suitable to the audit topic and objective(s).
Suitable criteria are relevant, reliable, objective, understandable, testable and complete. The
relative importance of these characteristics is a matter of professional judgement that should
be considered during the selection process. These attributes are shown in Figure 20.
96
Figure 202: Attributes of suitable criteria
Sufficient for the audit Contribute to conclusions that assist

purpose and do not omit decision-making by intended users and that
relevant factors or result in are linked to and answer the audit questions.
the omission of significant Relevant
information. Meaningful
and make it possible to
provide the intended users Complete Reliable Result in reasonably
with a practical overview consistent conclusions
for their information and when used by another
decision-making needs. auditor in the same
circumstances.
Possible to identify what

procedures and evidence
are needed to answer and Testable Objective Free from any bias on the
determine performance part of the auditor or the
against the criteria. audited entity.
Understandable
Clearly stated, contribute to clear conclusions
and are comprehensible to the intended users.
Not subject to wide variations in interpretation.
Source: GAO and GUID 3910: Central Concepts for Performance Auditing, 2019
According to ISSAI 3000/49, the auditor shall discuss the selected

Some criteria may be audit criteria with the audited entities as part of designing and
highly legitimate in the
eyes of the audited entity, such conducting the audit. Doing so helps to ensure a shared and
as its own goals, policies or common understanding of what criteria will be used as benchmarks
procedures. However, you must
be critical when considering the when evaluating the subject matter. It can help address questions
suitability of such criteria and regarding their legitimacy and applicability. Such discussion can be
not assume that the entity
meeting its own standards is a especially helpful in instances where you select criteria different
sign of good performance. than those used by the audited entities to measure their own
Organisations may set internal
standards too low in order to performance. However, while transparency and obtaining relevant
achieve them. In such cases,
input from the audited entities are important, it is ultimately the
look for additional criteria to
assess performance, such as auditor’s responsibility – not the audited entities’ – to select suitable
relevant benchmarks used in
the sector.
criteria for the audit based on the nature of the audit and the audit
questions.
97
Figure 21 shows examples of audit criteria in relation to their corresponding audit questions.
Figure 21: Criteria and corresponding audit questions
Audit question Criteria
To what extent is the environmental management Under the Water Act, the agency is required to
agency meeting the requirements of the Water prepare an annual report to Parliament on the
Act? operation of the Act.
To what extent is the education agency meeting The agency has established internal timeframes
timeframes for awarding contracts? for awarding different types of contracts.
To what extent has the agricultural management The agency’s policies require that processes be
agency established processes to ensure that established to determine the eligibility of
assistance payments are properly awarded? potential payment recipients and recoup any
monies erroneously awarded.
To what extent is the health agency ensuring that Water testing regulations prescribe the type and
potable water providers are performing water required outcomes of tests on water intended for
quality testing, as required? human consumption.
To what extent is the information technology Technically-developed standards dictate steps

office of the defence agency taking steps to the agency should follow to safeguard sensitive
safeguard sensitive employee information? employee information.
To what extent has the justice ministry’s grant Programme goals describe desired outcomes
programme contributed to desired outcomes and associated measures to assess progress in
within the target population? achieving such goals.
To what extent has the environmental agency’s Carbon emissions reduction targets specify the
carbon reduction programme achieved target desired levels of reductions across a period of
reductions in carbon emissions? five years.
Are there established mechanisms to raise According to the 2030 Agenda reference guide,
stakeholders’ awareness and ownership of the the countries should establish various
SDGs and 2030 Agenda in the country? communication strategies to engage/inform
different segments of the society as well as
integrate the public.
98
As noted earlier, sometimes criteria do not exist, are not appropriate
If you choose to develop
or are not readily measurable. In such cases, you may adapt or develop criteria, be sure to
new criteria. New criteria are usually not created from scratch; rather, consult with internal and external
stakeholders, as appropriate.
they are often derived from existing criteria, existing principles of social Also, consider the time and staff
science research or standards of professional practice. For example, resources to develop and
validate new criteria and
you can: look for and potentially adapt existing criteria used in similar whether they are proportionate
audit topics or operations; review existing literature and identify the to the audit’s overall cost and
importance.
measurement criteria used by experts in the field; or meet with
For example, if you are
officials, experts, consultants or focus groups to determine
developing criteria based on
performance benchmarks based on circumstances and comparable expert opinions, it is especially
important that the process of
practice, including in the international environment. When you selecting them is transparent
develop criteria, they need to be valid and convincing to a reasonable and defensible. Factors to
consider as part of this process
reader. Validating the criteria you develop is usually accomplished by include:
obtaining the views of independent, experts broadly representative of
• how the audit team will
the field. identify and collect information
from the experts (for example,
It is important to also discuss the criteria with the audited entities, panels, surveys, focus groups,
explaining why the additional criteria were needed and how they were etc.);
validated. The audited entities may have views regarding their • the appropriate mix of experts
applicability or identify other relevant information regarding the to ensure the desired mix of
perspectives, organisations or
appropriateness of the criteria you may not be aware of. It is also sectors;
helpful to obtain the audited entities’ feedback on the use of the • the experts’ certifications,
criteria, as it may increase the likelihood that the entities will agree reputation and experience;
with the findings and recommendations of the report and take actions • the experts’ actual and
accordingly. Should the audited entity not agree with the criteria you perceived level of
independence based on
selected, you may wish to involve third party experts to reconcile the potential conflicts of interest
different perspectives. However, while engagement and feedback relating to position, affiliation,
assets, sources of income and
from the audited entities are important, remember that it is ultimately other relevant circumstances;
the audit team’s responsibility to develop suitable criteria. Accordingly, and
in sustained disagreement, the audit team may choose to retain its • factors the expert will consider
criteria and disclose its rationale in the audit report. in forming their opinion.
99
How do you develop the audit methodology?
The Standard
During planning, the auditor shall design the audit procedures to be used for gathering sufficient and
appropriate audit evidence that respond to the audit objective(s) and question(s).
Once you have determined your audit objective(s), questions, criteria and scope, you will need
to consider what methodologies are appropriate for your audit, as well as the time and resources
available. Your methodology has to describe how you will collect and analyse information to
answer your audit questions. You can use a range of methods, the most common of which are
discussed in Figure 22.
100
Figure 22: Benefits and considerations of select information collection methods
Method Benefits Considerations
• Enables in-depth understanding of the • Needs to be carried out thoughtfully to
Interview
interviewee’s perspective. ensure consistency and enable
Discussion with one or
• Can be oftentimes set up and comparison.
more people, by phone,
completed relatively quickly. • Does not support statistical analysis.
internet or in person, to
• Enables information collection on • Takes time to identify and analyse
obtain their perspectives
sensitive topics. patterns or trends across several
on a programme or
• Can allow flexibility to quickly pursue interviews.
activity.
information in response to statements
made during the interview.
Document collection • Generally considered to be more • Source integrity, authenticity, authority,

Review of documents reliable than testimonial evidence and reliability must be carefully
gathered from the collected during interviews. considered (more info on Chapter 5).
audited entity and other • Usually provides good depth and range • May encounter difficulty gaining access
sources. of information. to information wherein the audited entity
does not readily provide documentary
evidence.
Direct observations and • Allows you to directly observe the • Observations intended to directly or
inspection programmes, people, property or events partially answer your audit question(s)
Physical observation of related to your audit. may be complex.
programmes, people, • Can provide context for the issues • Requires detailed planning and careful
property and events related to the audit. scheduling.
related to the audit to • The observation could affect the
collect qualitative behaviour of the person or situation
information. being observed.
• May require significant resources for
travel and staff participation.
Surveys • Way to gather information from multiple • Resource and time intensive.
Approach to information people. • Requires careful planning and testing.
or data collection that is • Data can be used for different types of • Can require time consuming analysis.
used to collect evidence analysis.
from a population using a • Data on selected variables may be
standard set of questions. generalisable and precise.
Site visits • Can combine different methods • Requires detailed advanced planning
Involves travel to a including interviews, document review and careful scheduling.
geographic location to and direct observations or physical • May require significant resources for
perform audit methods. inspections. travel and staff participation.
• Can improve cost efficiency by • Requires detailed understanding of how
combining multiple methods during one the audited entity or subjects of the
visit. visit(s) are organised.
File reviews and structured • Results in a structured and reliable data • Requires significant time and resources
observations set that can be used to support to execute this approach.
Information or data quantitative or qualitative analysis. • Requires detailed advanced planning
collection instruments used • Effective tool for collecting the data and the development of valid data
to systematically record needed to assess compliance with legal
collection tools.
observations and or regulatory requirements. • May not enable the determination of
information extracted from • Can provide data that may be the cause of identified deficiencies.
records. generalisable to a programme or
population.
Small group methods • Discussion can reveal issues not • Can be costly in terms of travel or fees to
Collection of information from addressed in individual interviews. convene expert panels.
a group of people using tools • Adaptable for a variety of audit needs. • Analysis can be difficult and time
like focus groups (facilitated • Experts can provide consensus consuming due to volume and diversity
small group conversations) perspectives on issues or activities. of information.
and panels of experts.
101
Secondary data • May be faster than other data • Data may not match the audit
Data collected by collection methods. objective.
someone else, such as • Data may be more complete than if • Data may be difficult to access.
government agencies, you collected it yourself. • May require significant time to assess
universities or research • Quality checks may have already been the reliability of the data.
organisations. completed.
Case studies • Can enable in-depth assessment of • May require substantial time and
In-depth collection of activities, facilitate the analysis of resources.
data for one or more similarities and differences between • Analysis can be time-consuming.
complex events, operations in different localities, or • Case study selection will significantly
incidents or locations illustrate aspects of processes or the impact information collection and
that seeks to answer consequences of flaws in programmes findings.
complex ‘why’ or ‘how’ using specific ‘real-world’ examples.
questions. • Enables collection of more in-depth
information about a topic or complex
events.
• Approach can enable corroboration of
evidence and increase the reliability
and validity of findings.
Once information and data are identified, you will need to give some thought during the
planning phase to how you intend to analyse the data. There are some analysis methods that
you can consider in developing your methodology, including those discussed in Figure 23.
102
Figure 23: Benefits and considerations of select data analysis methods

Content analysis • Enables the identification of patterns • Labour and time intensive.
Method for structuring or trends in data that are meaningful • Can lead to unusable results if
and analysing complex for the audit questions and implemented incorrectly.
qualitative information objectives. • Requires planning and training of staff.
obtained from multiple • Allows data to guide the
sources, such as development of analytic categories.
interviews or a large • Can be used to inform the use of
number of documents. other methods.
• Enables unstructured data to be
summarised, analysed and reported.
Statistical analysis and • Enables the identification of patterns • Requires significant expertise in the use of
modelling and correlations in large quantities of data analysis software.
Use of software and data. • May require significant time and resources
computer models to • Provides an efficient and structured to structure the data so that it can be
identify trends, patterns means of analysing large amounts of analysed using data analysis software.
and correlations in large quantitative data. • Typically does not identify the cause of
data sets. patterns or correlations.
When designing your data collection and analysis methods, you will need
Remember, although
to ensure the approaches you use will enable your team to obtain evidence you will make initial
that addresses your audit objective(s) and answers your audit questions. decisions about your audit
methodology during the
Additionally, you will need to consider risks and limitations that result from planning phase, you may need
your team’s expertise, cost and time limitations, and the availability and to refine or adjust your
methodology as you perform
reliability of the data (see the Managing risk section below for more the audit. This will be discussed
information). In most instances, you will find it beneficial to use multiple in greater detail in Chapter 5.
methods to collect and analyse data to help you corroborate information

from multiple sources.
103
Examples of applying audit methods
The Packaging Recycling Obligations; 2018. Federal Monitoring and Evaluation Guidelines
In completing this audit, the audit team used a Incorporate Most but not All Leading Practices;
number of information collection and analysis 2019.
methods including: In completing this audit, the audit team used a
number of data collection and analysis
1. Interviews with stakeholders to understand methods including:
their view of the purpose of the scheme, its
performance, the level of fraud and error 1. Content analysis drawing upon
present, and the oversight government has had documentation and interviews to agency
over it. guidelines against 28 leading practices.
2. Direct observation of the Environment
Agency’s central packaging compliance team 2. Content analysis and document review of
to develop an understanding of how the agency policies, guidance and operating
scheme operates and is monitored. procedures against requirements established in
3. Content analysis of legislation, policy papers, government guidelines.
department briefing notes, audit reports,
industry reports and relevant published audit 3. Interviews with agency officials.
reports.
4. Statistical analysis of the National Packaging
Waste Database to determine the number of
companies registered and accredited with the
scheme, the amount of revenue reported as
generated through the system and trends in the
reported weight of packaging recycled.
Ultimately, you will need to not only consider how you will collect evidence and how you will
analyse it to address your audit questions but also how you will assess the evidence to ensure it
is reliable. Collectively, these steps establish the methodology for your audit, something we
discuss in greater detail in Chapter 5.
How do you manage risk during audit design?
The Standard
The auditor shall actively manage audit risk to avoid the development of incorrect or incomplete audit
findings, conclusions and recommendations, providing unbalanced information or failing to add value.
It is important to manage risk throughout the audit design process. A key purpose of audit design
is to identify, mitigate and plan for major risks; accordingly, all design decisions have to be risk-
based.
104
Audit risk is the possibility that the auditors’ findings, conclusions or recommendations may be
incorrect or incomplete due to factors such as inadequate audit processes, insufficient or
inappropriate evidence, resource or data limitations, or intentional omissions or misleading
information because of misrepresentation or fraud (GUID 3910/61). This includes the risk that
auditors will not detect a mistake, inconsistency or significant errors – or fraud in the evidence
supporting the audit. Risk involves the probability of an event occurring combined with the
seriousness of the event if it occurs.
How do you identify and assess risk?
Identifying and assessing risk during audit design requires sound, up-to-date knowledge of the
audit area, including a thorough understanding of the audit topic objectives, policy and
processes, along with key stakeholders and controls. The identification of audit risk involves
consideration of both qualitative and quantitative factors, including time frames, complexity and
sensitivity of the work; the size of the activities in terms of financial value and number of citizens
served; adequacy of the audited entities´ systems and processes for preventing and detecting
inconsistencies, significant errors or fraud; and auditors’ access to records.
You should identify and assess risks for the audit overall and each potential audit approach so
that you have a clear understanding of the costs, benefits and limitations of potential
methodologies. Risk identification and assessment can take many forms but may generally be
addressed by considering the following questions (GUID 3920/61):
• Does the audit team possess sufficient skills and knowledge for the audit (including specialised
knowledge for specific tasks)?
• Are the time frames and resources needed to conduct the audit available and feasible (for
example, travel funds, opportunity cost impact on other audits)?
• Is the audit topic sensitive, highly visible or controversial (for example, political sensitivity,
media sensitivity)?
• Is the audit and subject matter highly complex, or does it involve areas traditionally prone to
risk (for example, IT systems, procurement, health and environmental issues)?
• Are there real or perceived threats to the independence of the auditors assigned to the audit?
• Is there risk related to management integrity or relations with the audited entities?
• Are there enough data available and are the data of good quality (for example, data access
and reliability)?
105
In identifying and assessing risk, you may benefit from evaluating
whether the audited entities have taken appropriate corrective action Plan the steps you will
take to assess data
to address findings and recommendations from previous audits that quality. Possible steps include:
are significant in the context of the current review. This information
• reviewing information about
can be used to determine the nature, timing and extent of current the data from reports, studies,
system manuals and
audit work, including how testing the implementation of corrective
knowledgeable parties;
actions applies to the current audit.
• testing the data (for example,
checking the total number of
Once you are aware of risks, you have to carefully consider your risk records, testing for missing values
or elements, looking for invalid or
tolerance – that is, the acceptable level of variation in audit duplicate records and following-
performance relative to the achievement of your audit’s objectives. up on anomalous data such as
extremely high values or dates
Risk tolerance should also be balanced against the benefits of outside of valid time periods);
undertaking the task. For example, if conducting a survey, you need to
• assessing internal controls of
consider your tolerance for risks – such as a low response rate or the data system; and
limited access to staff with the expertise needed to properly design and
• tracing a sample of data to
administer the survey – about the potential benefits of the survey. the source documents to ensure
accuracy.
When determining your tolerance for risk, focus on the risks most likely See Chapter 5 for additional
to affect the audit’s critical path, which comprises the tasks that will information on assessing data
quality.
delay the completion of the project if they are not performed as
expected and on time.
Key risks can be Your approach to assessing risk during the audit design phase can vary and
captured in the
design matrix, and sufficient is a matter of professional judgement, depending on the audit’s
time should be included in circumstances and approach. See Appendix 5 for tools that can enhance
the project schedule or work
breakdown structure for risky knowledge of the subject matter and facilitate the analysis of audit risks.
tasks and to further assess risk
as the audit evolves. The
project schedule can also
help you determine which How do you mitigate audit risk?
tasks are critical and
therefore most in need of
mitigation. See the ‘How do After identifying and assessing audit risks and tolerance levels, it is important
you document the audit
plan?’ section for additional
to manage any significant risks by planning steps to reduce them or mitigate
information on these their effects (GUID 3920/62). This can be accomplished through various
important design tools.
actions, including:
106
• increasing or reducing the scope of work;
• adding specialists (for example, methodologists), reviewers or additional senior staff;
• increasing resources;
• regularly monitoring or tracking progress against interim milestones by updating audit plans,
holding meetings or producing status reports;
• building in extra time, if possible, for particularly risky tasks;
• changing the method to obtain additional evidence, higher-quality evidence or alternative
forms of corroborating evidence;
• aligning the findings and conclusions to reflect the evidence obtained; and
• increasing supervisory or management review.
For example, if your team has concerns about data quality, you could plan to mitigate the risks
associated with its use by: collecting additional evidence from other sources to supplement or
corroborate the data; and including information in the report about the source and quality of the
data, along with any associated limitations in its use or interpretation. Remember, you should
only use data that you determine to be sufficiently reliable for the intended purpose of your
audit.
When considering ways to mitigate risks, remember that risks and Your plans for mitigating
mitigating steps associated with audit approaches should always be key risks can be captured
in the audit design matrix, which
balanced against the benefits of those approaches in order to clearly is described in the ‘How do you
understand their value and optimise the return on invested resources. document the audit plan?’
section.
Chapter 2 discusses the broader process of managing risk across the
entirety of the audit.
How do you determine the time frames and resources needed for a performance
audit?
When designing your audit, it is critical that you determine realistic time frames and resource
needs so that the work can be performed in an economical, efficient, effective and timely
manner, in accordance with the principles of good project management. To perform a high-
quality audit within a limited time frame, it can be helpful to think of the audit as a project
because it involves planning, organising, securing, managing, leading and controlling resources
to achieve specific goals. In particular, this requires that you:
• determine realistic time frames for the audit and individual tasks that need to be completed.
These have to be based on the planned methodology and other relevant factors, including
107
internal audit processes, past audits, stakeholder perspectives, anticipated access to
information, and the availability of resources;
• identify and align a sufficient number of auditors, supervisors, and internal and external
stakeholders with specific tasks to meet expected time frames for completing the work. This
process has to account for their collective knowledge, skills, abilities, independence and
developmental needs. See Chapter 2 for additional information on ensuring audit team
competence; and
• determine costs associated with travel, training, equipment and external subject matter
experts, and other ancillary costs. Internal staff resources are typically budgeted in terms of
working days and tracked through an internal recording system, whereas external
stakeholders may involve separate costs.
How do you document the audit plan?

What is an audit plan?
It is important that auditors prepare a written audit plan to guide their work and ensure the audit
is properly designed (see GUID 3920/56-58). The intent of an audit plan is to synthesise and
document the design efforts discussed earlier, tying together all design considerations and
components. The form and content of an audit plan may vary among audits but often includes a
design matrix, project schedule and any other appropriate audit documentation of key decisions
about the audit objectives, scope and methodology, and the auditors’ basis for those decisions.
This could include a SWOT analysis (Strengths, Weaknesses, Opportunities and Threats) and Risk
Verification Diagram (RVD), the results of the audit pre-study and data collection plans and tools,
among other items (see Appendix 5). Collectively, these items should encompass:
• background knowledge and information needed to understand the subject matter and the
entities being audited;
• the audit objective(s), questions, criteria and scope, including the period to be covered;
• results of the risk assessment;
• methods for gathering evidence and conducting audit analysis;
• the plan for conducting the work, including key tasks, time frames, milestones, resources
(including team members and need for external expertise) and control points; and
• the estimated cost of the audit, with or without staff costs depending on the planning system
of the SAI.
A written audit plan provides an opportunity for your SAI management to supervise audit design
and to determine, among other things, whether: the proposed audit objectives and questions
108
are likely to result in a useful report; the audit adequately assesses risks; the proposed scope and
methodology are adequate to address audit objective(s), and the available evidence is likely to
be sufficient and appropriate for the audit.
The plan is also a tool to help management determine whether sufficient staff, supervisors and
specialists with adequate collective professional competence and other resources can conduct
the audit and meet expected time frames. Therefore, it is important to submit audit plans to SAI
management for approval, as discussed in the How do you involve internal stakeholders, external
stakeholders and management when designing the performance audit? section below. The
approved plan will then guide your team in the audit and provide the basis for management to
regularly monitor its progress. To do so effectively, the plan has to allow for flexibility so it can
be adjusted as circumstances change and knowledge deepens during the audit.
How do you develop the design matrix?
The audit design matrix is a key tool for providing an overview of and documenting the audit
design. It provides a structure for synthesising and linking the elements of your audit design,
enabling a more systematic and directed design process, as well as communication with internal
stakeholders within your SAI about the audit approach. The design matrix helps document and
link your audit scope, objective(s), criteria and methods, assuring a logical chain of reasoning
between the audit’s approach and likely results. It provides the basis for stakeholders to develop
a common understanding of the audit’s design and ultimately agree on the planned approach.
While the matrix is initially to be prepared during the design phase, it is a living document
reviewed and updated, as necessary, as the audit work progresses.
The main goals of the design matrix are to:

• document and formalise the audit approach;
• present a summary overview of the audit design;
• identify and document the ‘why, what, and how’ of the work by establishing a clear
relationship between the audit’s scope, objectives and methodology;
• link the work performed to its expected results; and
• facilitate stakeholder interaction, audit supervision and review.
When preparing the matrix, make sure to explicitly identify the intended users of the report so
that their needs and interests can be considered. Doing so will help ensure the report is useful
and understandable to its intended audience. However, such considerations should in no way
undermine the independence and objective attitude of the audit team, which remains
responsible for conducting a well-balanced and independent performance audit.
109
Figure 24 provides an example design matrix template, along with instructions for completing
each section.
Figure 24: Design matrix template
• Put the issue into context; state why it is important.

• State why the audit is being conducted.
• Identify the audit team and intended users of the audit report.
• Introduce the overall audit objective(s).
Audit Criteria and Scope and Limitations Expected

question(s) information required methodology results of the
and source(s) including data work
reliability
Identify key audit Criteria: Identify the Scope: Identify the Identify any limitations Describe the
questions. criteria or plans to planned scope of the associated with the expected results of
collect documents work associated with information required, the work by
Audit questions may that will establish the the research planned summarising what the
be descriptive or criteria to be used. objective. methodology or your audit team will likely
evaluative. Scope will define the general ability to be able to say as a
As discussed, this can boundaries or time answer the audit result of the work
Ensure each question include laws, frame of your work for question. Limitations performed.
is specific, objective, regulations, policies, the objective. could include
neutral, measurable best practices or questionable data The expected results
and doable. Ensure other credible Methodology: quality or reliability, should answer the
key terms are defined. standards for how Describe strategies for inability to access audit question in the
things should be. collecting required some information, first column.
Broad questions information or data, constraints on staffing
followed by more Information required such as document or travel funds, or
pointed sub-questions and sources: Identify review, data inability to generalise
sometimes help to the information collection instruments, or extrapolate
clarify scope and required to answer questionnaires, focus findings to the
develop more the audit questions groups and case universe.
substantive findings. and the sources of this studies. Address the
Limit the number of information, including planned scope of Discuss how each
sub-questions to no documents, each strategy, limitation may affect
more than three. programme officials, including time frames, the product and
databases, subject locations and sample describe steps to be
As the audit nears its matter experts, etc. sizes. taken to mitigate the
conclusion, audit associated
questions may be When the first column Describe the challenges.
refined to reflect your contains sub- analytical techniques
findings more questions, precise to be used to analyse If the limitations are so
accurately. one-to-one linkage is the information severe that they will
not strictly necessary. collected, such as materially affect your
Consider what it will content analysis, case ability to answer the
take to answer the study summaries or audit question,
question and avoid regression analysis. consider rewording
repetition. the question and/or
Describe steps to be altering the scope to
taken to assess the decrease that risk.
reliability of data
sources.
Source: US GAO
The design matrix can also be documented in other formats. One such format is the design paper,
which presents the same information in narrative form outside a structured matrix. The design
110
paper itself can take multiple forms, depending on audit circumstances and staff/management
preferences. If used, see the checklist in Appendix 6 to help ensure your design paper includes
the necessary information.
How do you develop the project schedule and a work breakdown structure?
The project schedule and work breakdown structure create a roadmap for performing the work
and answering the detailed questions of ‘how’ the work is being conducted, ‘when’ the work will
be conducted and ‘who’ will conduct the work. Like the design matrix, the project schedule and,
if used, a work breakdown structure (a work breakdown structure is not always necessary) has
to initially be prepared during the audit design phase. However, since the auditing process is not
static, you have to continuously monitor your schedule and work breakdown structure and take
corrective actions, when appropriate, to ensure the plans reflect the work being performed and
that the audit proceeds in an efficient manner.
Collectively, the project schedule and work breakdown structure will help you define and
document:
• the specific tasks the team will perform;
• when tasks will occur (timing and sequence) and how long they will last;
• how the tasks relate to each other;
• who is needed and available, and for what periods;
• other required resources (for example, travel funds, training costs);
• milestone dates (that is, key decisions or progress assessment dates); and
• the detailed activities associated with each major task.
The project schedule and work breakdown structure are similar tools, but they provide different
types of information and varying levels of detail. Specifically, the project schedule – which is
typically developed for all audits – focuses on the audit’s key activities, durations and associated
staff, allowing you to define and sequence audit tasks, allocate resources and closely monitor
their usage. Alternatively, the work breakdown structure allows you to divide the work into
distinct increments and describe the tasks that will be performed to the level of detail necessary
to define the scope of work and enable its oversight. Unlike the project schedule, the work
breakdown structure generally does not emphasise time frames associated with the work. This
may be particularly useful when you need to define in detail the work associated with a major
line of effort, such as developing a survey and focus oversight on the execution of specific
detailed steps instead of overall timeliness. Whether you choose to use a work breakdown
111
structure or just the project schedule, it is important to carefully monitor audit progress, along
with the expenditure of staff time and budgeted resources.
See Appendix 7 for templates, examples and detailed descriptions for the project schedule (basic
and detailed variants) and work breakdown structure.
How do you involve internal and external stakeholders and management when
designing a performance audit?
Effective communication with internal stakeholders (that is, technical experts, legal experts,
methodologists) and your management, as well as external stakeholders – such as the audited
entities, legislature, the media and other concerned actors – is essential in order to properly plan
and conduct your audit.
How do you communicate with internal stakeholders and management?
Your ability to develop and maintain a sound audit plan depends to a large degree on the extent
to which you communicate with internal stakeholders and SAI
Key decisions, management throughout the initial and ongoing design processes. As
communications and
changes made to the audit plan discussed, your audit plan needs to be developed in conjunction with
need to be documented in the internal stakeholders and submitted to SAI management for approval. As
audit plan or other
documentation, as appropriate. part of this process, it is helpful for the audit team, supervisor, internal
stakeholders, and management to collectively discuss and reach an
agreement on the audit plan, as documented in the design matrix, project
schedule and other chosen tools. Doing so will help ensure all parties agree on the approach and
accept the audit risks that may exist because the audit plan has not yet been tested. Often this is
accomplished through a formal meeting that is required by SAI policy.
As discussed throughout this chapter, design is a continuous process. It is therefore important

that you plan to have regular meetings with your management to inform them of audit progress
and the use of assigned resources. This will allow management to guide any necessary changes
to the audit plan and continuously ensure that assigned resources are adequate to successfully
conduct the audit. Similarly, it is equally important that you plan to consult often with internal
stakeholders, drawing upon them for their expertise. This can be accomplished through periodic
meetings, milestone discussions, status checks and ad hoc consultations.
112
As the audit unfolds, your ongoing communication with both stakeholders (such as
methodologists and legal experts) and management should focus on the execution of the audit
plan and the emerging preliminary findings. Accordingly, tools such as the project schedule,
design matrix and work breakdown structure provide mechanisms for coordinating continuous
stakeholder and management involvement.
How do you communicate with external stakeholders?
The Standard
The auditor shall plan for and maintain effective and proper communication of key aspects of the audit
with the audited entity and relevant stakeholders throughout the audit process.
The auditor shall take care to ensure that communication with stakeholders does not compromise the
independence and impartiality of the SAI.
When designing your audit, it is important that you also communicate with external stakeholders
– which include the audited entities, the legislature and other relevant government offices – and,
when appropriate, non-government stakeholders such as the media.
Communication with the audited entities should begin during the audit planning stage and
continue throughout the audit process (GUID 3910/64). It is important that you engage the
audited entities early to discuss the audit subject matter, objective(s), criteria, audit questions
and information needed, along with the period to be audited and the government activities,
organisations and/or programmes to be included (GUID 3910/65). Discussing these key aspects
will provide a clear picture of what the audit is about and why you are doing it, what the result
might be, and how the audit will affect the audited entities. Further, it creates a basis for
exchanging views, avoiding misunderstandings and facilitating the audit process. This does not
mean that the audited entities dictate conditions or in any way control the audit process. Rather,
it helps establish a constructive process for interacting with the audited entities that are essential
to performing an effective audit. (GUID 3910/66)
Determining the form, content, and timing/frequency of communication with management or

those charged with governance of audited entities is a matter of professional judgement.
However, a combination of written communication and in-person meetings are generally
preferred. For example, you may wish to use a letter to inform the audited entities of key
information as the audit is initiated and hold a meeting to discuss key aspects of the audit, as
113
discussed. Additionally, some SAIs prefer to provide the audited entities with detailed
information on the design of the study as early as possible to help reassure the audited entities
about the nature and scope of the audit, while other SAIs prefer to provide such information only
after the audit plan has been approved by management. Organisations accustomed to working
with SAIs and participating in the audit process may have established protocols they want you to
follow when working with them. For example, audited entities may want you to send requests
for information through specific points of contact. Similarly, many SAIs have established
protocols that clearly define policies and practices for how you are to engage with the entities
you are auditing. The exact timing of such communications is a matter of professional judgement
and the requirements of your SAI; however, it is helpful to consider providing the audited entities
with as much information as early as possible to develop a mutual understanding of the audit’s
purpose and scope.
Communication with other external stakeholders during the design phase is shaped by each
party’s role, needs and interests, and internal SAI protocols. For example, if the audit is being
conducted at the request of the legislature, it may be helpful to contact the requesters when
initiating the audit in order to obtain clarifying information, follow-up to explain the audit design
and schedule, and provide periodic briefings on the status of the audit and preliminary findings.
It is also important to gain the trust of the audited entities to ensure cooperation throughout the
audit. See GUID 3910/70-73.
While communication with the media generally occurs after an audit report is issued, the SAI or
audit team may need to be prepared during the design phase to respond to media enquiries or
even develop a strategy for engaging the media as needs dictate, such as when the audit topic is
controversial or high-profile. For ongoing work, it is generally appropriate to share only a limited
amount of information with the media such as: the audit objective(s), scope and methodology;
the source of the work; and the expected completion timeframe. Audit details or potential
findings are usually not shared with the media until work is completed and the audit report is
issued.
114
When designing a performance audit, remember to...
… communicate with the audited entity and … consider the independence of audit team
other knowledgeable actors to obtain the members to ensure that work plans are
information necessary to develop a sound objectively constructed and can be soundly
audit plan; executed;
… develop sufficient understanding of the … assess the risks associated with different audit
audit area, weaknesses and challenges in it, questions and methodologies (for example, time,
what data will be available during the audit, data quality) and take appropriate mitigating
the materiality of the audit questions and what steps (for example, adding reviewers, obtaining
criteria will be considered for assessing corroborating information) to ensure that efforts
performance; will produce findings, conclusions or
recommendations that are accurate, complete
… consider resource availability and audit
and add value. Remember that all design
team competence when determining the who,
decisions are risk-based decisions;
when and how work will be conducted;
… apply professional judgement to all planning
… communicate continuously with internal
decisions to ensure sound decision-making
stakeholders (and external stakeholders as
based on relevant factors; and
appropriate) to ensure the audit plan reflects
legal, subject matter and methodological … document key planning considerations and
expertise; decisions via tools such as the design matrix,
project schedule and work breakdown structure.
115
Chapter 5
How do you conduct a performance audit?

Conducting the audit
• Understand the importance of collecting • Analyse the collected information and data
sufficient and appropriate evidence. using qualitative and quantitative methods.
• Gather information and data by employing
the approved methodology.
The purpose of conducting a performance audit is to obtain sufficient and appropriate evidence
to develop findings that answer the audit objective(s) and questions. As discussed in Chapter 4,
the audit questions should guide your audit work; thus, the information you collect and analyse
should directly address the audit questions.

• How do you determine the sufficiency and appropriateness of evidence?
• How do you gather information for a performance audit?
• How do you analyse information?
• How do you document and safeguard information?
These activities can occur sequentially or concurrently, depending on the audit and the types of
methodologies your team has decided to use. In practice, information is often collected, analysed
and evaluated for sufficiency and appropriateness simultaneously. It can also be helpful to begin
to identify the elements of potential findings while you are still collecting data. Doing so can help
you identify any gaps in your evidence and the need for additional data collection. This is usually
an iterative process.
During data collection, your audit team may also need to revisit some of the decisions made
during the planning phase of the audit. For example, as you identify new potential sources of
116
information that can be used as evidence or if you determine that some of the information
collected is not reliable or helpful in answering the audit questions, you may need to adjust the
audit scope, questions, the application of criteria, and methods for information collection and
analysis. Remember to obtain your management’s approval for any material changes to your
audit plan and keep your internal stakeholders and the audited entities informed. (GUID
3920/44-47, 72)
How do you determine the sufficiency and appropriateness of evidence?
The Standard
The auditor shall obtain sufficient and appropriate evidence in order to establish audit findings, reach
conclusions in response to the audit objective(s) and audit questions and issue recommendations when
relevant and allowed by the SAI’s mandate.
Audit findings must be supported by evidence, so the quantity and quality of the evidence you
obtain is important. This means you will need to continuously consider and evaluate the evidence
you are: (1) planning to obtain; (2) are in the process of obtaining; or (3) have already obtained,
for sufficiency and appropriateness (GUID 3920/69-77). Before we present various methods to
collect and analyse information and data, it is important to understand the differences between
information and evidence. When qualitative and quantitative information is collected that can
be used to support a point you wish to establish related to the audit questions; it becomes audit
evidence. Though all the information collected during the audit can help you develop your
understanding of the audit topic. Often the evidence you will use to support your findings
emerges through your analysis of the collected information.
Sufficiency refers to the quantity of evidence collected (see Figure 25). Do you have enough
evidence to persuade a knowledgeable person that the findings are reasonable? For example,
information obtained from only one source, such as an interview or a single document, will likely
not be enough to support a finding but may still be relevant to use as a general illustration. It is
important that findings be supported and corroborated by multiple sources and types of
evidence.
117
Figure 25: Sufficiency of evidence
Sufficiency is a measure of the quantity of the evidence you

use to support findings and conclusions related to your audit
objective(s) and questions.
Have you obtained enough evidence to persuade a

knowledgeable person that the findings are reasonable?
How much evidence is sufficient depends in part on the appropriateness of the evidence?
Appropriateness refers to the quality of the evidence. Is the evidence relevant, valid and reliable?
It is important to consider the source, content, and timing of your evidence when making these
determinations. Figure 26 contains more information on these important concepts.
Figure 26: Appropriateness of evidence
Appropriateness
Relevant evidence has a logical relationship with, and

importance to, the issue being addressed. For example, if you
are auditing the procedures for customs inspections at airports,
information about the parking procedures at the airport would
not be relevant.
Valid evidence is based on sound reasoning or accurate

information. For example, information obtained from the
website of a political party may not be a valid source of
evidence because the source of the information could be
biased.
Reliable evidence means results are consistent when

information is measured or tested and must be verifiable or
supported. For example, quantitative data that you obtain
from an information system may not be reliable if you find that
users do not enter the data into the system consistently or
check it for errors. Evidence collected from different sources
and at different times should be consistent.
118
You need to obtain your data from knowledgeable and reliable sources using accepted methods.
In performance audits, evidence will typically be persuasive (that is, pointing toward a
conclusion) instead of conclusive (that is, definitively stating ‘yes/no’ or ‘right/wrong’) (GUID
3920/71). Ultimately, determining whether you have sufficient and appropriate evidence for
your findings will require professional judgement. In making such determinations, you will need
to be aware of the potential strengths and weaknesses of your evidence and consider the source
of the evidence, as some sources may be more credible or reliable than others. Find below useful
tips to consider when assessing the sufficiency and appropriateness of your evidence.
119
Sufficiency and appropriateness of evidence
Sufficiency Appropriateness
✓The greater the audit risk, the greater the ✓ Ensure that your evidence is relevant – that is, of
quantity and quality of evidence required. importance to your audit topic.
✓The more important the finding, the greater ✓ Ensure that your evidence is valid – that is,
the quantity and quality of evidence based on accurate information and logical
required. analysis.
✓Stronger evidence may allow less evidence ✓ Ensure that your evidence is reliable – that is,
to be used. results are consistent and able to be verified.
✓Having a large volume of audit evidence ✓ Documentary evidence is often more reliable
does not compensate for a lack of than testimonial evidence, but the reliability
relevance, validity or reliability. varies depending on the source and purpose of
the document.
✓More evidence is normally necessary when
the audited entity(ies) or other stakeholders ✓ Testimonial evidence that is corroborated in
have different opinions on the subject writing is more reliable than oral evidence
matter. alone.
✓ Evidence-based on many interviews is more
reliable than evidence based on a single or a
few interviews.
✓ Testimonial evidence obtained under
conditions in which people may speak freely is
more valid and reliable than evidence
obtained when people may feel intimidated.
✓ Evidence obtained from a knowledgeable,
credible and unbiased third party is more valid
and reliable than evidence obtained from the
management of the audited entity or others
who have a direct interest in the audited entity.
✓ Weak internal controls can affect the reliability
and consistency of evidence across an
organisation. Thus, evidence obtained when
internal control is effective is more reliable than
evidence obtained when the internal control is
weak or non-existent.
✓ Evidence obtained through the auditor’s direct
observation, computation and inspection is
more reliable than evidence obtained
indirectly.
✓ Original documents are more reliable than
copied documents.
Source: Adapted from GUID 3920/75-76 and Government Auditing Standards (US GAO)
120
Thoughtfully assessing and ensuring the sufficiency and appropriateness of your evidence
throughout the audit is a critical responsibility of your audit team. It will require that you apply
professional judgement and critical thinking skills. (GUID 3920/77)
If you find limitations or uncertainties in your evidence, there are steps you can take to try to
mitigate the audit risks. These steps include:
• seeking independent corroborating evidence from other sources;
• presenting the findings and conclusions so that the supporting evidence is sufficient and
appropriate for the purposes used. You also need to describe in the report any related
limitations or uncertainties with the validity or reliability of the evidence if such disclosure is
necessary to avoid misleading the report users about the findings or conclusions;
• redefining the audit questions or the audit scope to eliminate the need to use the specific
evidence that is causing concern. Remember to inform the audited entities about any
significant changes; and
• determining whether to report the limitations or uncertainties as a finding, including any
related significant internal control deficiencies.
The results of your evaluation of the sufficiency and appropriateness of evidence and any
mitigations may not be clear cut, and you may have to make difficult determinations as an audit
team and with your management. When making these determinations, it is important to
remember that evidence is not sufficient and appropriate when:
• using the evidence carries an unacceptably high risk that it could lead you to reach an incorrect
or improper conclusion;
• the evidence has significant limitations, given the audit questions and its intended use; and
• the evidence does not provide an adequate basis for addressing the audit objective(s) and
questions or supporting the findings and conclusions.
As you move forward with your information collection, remember that a healthy scepticism
about what people tell you and the information from documents you obtain – not simply
accepting things at face value – is extremely important for you to do quality work. This is called
professional scepticism, and it is a key component of two audit concepts – independence and
professional judgement, as discussed in Chapter 2.
For example, as you collect testimonial evidence, it is important that you consider the credibility
of the people being interviewed – what is their position, knowledge, expertise and
forthrightness? Descriptions of the person’s actions and other people’s actions may or may not
be reliable, and it is therefore important that it be considered from all angles. For instance, there
121
are often tensions and different interests within an organisation, such as between departments
and between managers and staff. While this may motivate people interviewed to share
information with the auditors, it is imperative for the auditors to be mindful of these tensions
and assess the reliability of the information because it may represent vested interests rather than
fact.
Even when the person interviewed describes the situation with honesty or a document they
share with you addresses the audit topic, the information may not fully and correctly describe
the real situation because different people and organisations may have different perspectives
and preferences and thus interpret the reality in different ways. All individuals are experts on
their own role, perspective, knowledge and opinions – but may not know the full ‘story’ and may
not be able to see issues from other equally relevant perspectives. It would be extremely rare
that sufficient and appropriate evidence could be obtained from a single interview or document.
There may be specific circumstances where the individual being interviewed or the document
used is uniquely authoritative in relation to the audited activity, but it is important that you apply
considerable caution and professional judgement when evaluating such circumstances. Using
multiple interviews with staff in different positions and roles, on the other hand, can enable the
auditors to develop an understanding and analysis of the organisation going beyond what people
in it have been aware.
Keeping the sufficiency and appropriateness of the evidence in mind as you conduct audit work
will help you ensure that you have enough quality evidence to develop strong audit findings.
How do you gather information for a performance audit?

How do you work with the audited entities?
As with planning, gathering information will generally require you to coordinate closely with the
audited entities and any other organisations from which you will need to obtain information
(GUID 3910/63-69).
Below are some general tips for communicating with the audited entities as you conduct audit
work to help ensure smooth and efficient information collection.
122
Communicating with the audited entities
✓ Agree with the audited entities on the ✓ Give the audited entities sufficient time to
procedures that you will follow to schedule respond to your information requests. The
interviews and site visits and to request precise time frames will vary depending on
information to avoid miscommunication and the complexity of the request but understand
delays. A ‘no surprises’ approach is generally that large requests for information may take
wise. the audited entities additional time to pull
✓ Plan ahead! Recognise that the audited together.
entities are busy carrying out their primary ✓ Keep the audited entities informed of your
mission. The more advance notice that you progress on the audit and any significant
provide the audited entities about your changes to your audit plan and timeframes.
requirements for the audit, the better chance
you have of obtaining the information that ✓ Escalate early to your management any
you need within your desired time frames. challenges you encounter in obtaining
information from the audited entities so these
✓ Identify agreed-upon points of contact within issues can be quickly resolved.
the different offices at the audited entities to
facilitate direct and responsive ✓ Communicate and work to resolve these
communication. issues with the audited entities.
✓ Agree with senior management in audited ✓ Be professional, courteous, and fair in all your
entities on who you will keep informed about dealings with the audited entities.
the progress of the audit, making further
✓ Discuss emerging preliminary findings with the
dissemination of such information the
audited entities during the audit to get their
responsibility of the entity itself.
feedback and input.
✓ Notify the audited entities as early as possible
✓ Revisit audit protocols with the audited entity if
of the interviews and site visits that you plan
you encounter challenges or delays and
to conduct and within what general
adjust as necessary.
timeframes.
Chapter 4 discusses meeting with audited entity at the beginning of your audit. After the initial
meeting, during the planning phase, it is important to continue to communicate with the audited
entities throughout the audit about your planned work and time frames to ensure that the
officials understand the scope of the audit, your plans and your progress. Regular discussions
with the audited entities can be useful to identify additional sources of evidence or to obtain
perspectives that may inform the development of findings. It is also important for you to discuss
with the audited entities the methods your audit team will use to collect information so that the
audited entities are prepared to support your efforts.
Most audits will also include a meeting with the audited entities at the end of the audit. Your
audit team can confirm that the key facts support your findings and discuss your findings, and
any potential recommendations, with the audited entities. This meeting is sometimes referred
to as an exit conference. The exit conference is an opportunity for you to share a preliminary
draft of your audit report and discuss the audited entities’ perspectives on your preliminary
123
findings and recommendations, as applicable. It presents an opportunity for you and your team
to make any needed changes before providing the formal report to the audited entities for official
review and comment. These steps are discussed in more detail in Chapter 7.
A sound dialogue throughout the audit process with the audited entities is pivotal in achieving
real improvements in governance and may increase the impact of the audit. In this context, the
auditor can maintain constructive interactions with the audited entities by sharing preliminary
audit findings, arguments and perspectives as they are developed and assessed throughout the
audit (ISSAI 3000/58). Typically, you will not present the SAI’s findings to the audited entities until
the end of the audit – first at the exit conference and then when you publish a final report.
However, as you are conducting your work, if you find issues that require immediate corrective
action – such as evidence of fraud or significant internal control deficiencies that could lead to
fraud (see below) – it is important that you communicate these issues to your management as
soon as possible (GUID 3910/91-93). It is recommended that you also discuss with your
management how and when to inform the audited entities of these issues.
124
Fraud
Fraud involves an individual or entity obtaining or But you need to continuously assess the risk
attempting to obtain something of value through of fraud related to your audit objective(s),
wilful misrepresentation. including factors such as:
• individuals’ incentives or pressures to
For example, an entity that misstates or commit fraud;
misrepresents programme information or results to • the opportunity for fraud to occur; and
obtain government funding may be committing • attitudes that could increase the risk of
fraud. fraud.
As an auditor, it is not your responsibility to uncover If information comes to your attention during
fraud or to determine whether an act is fraud. This is the audit indicating that fraud, significant
the responsibility of a judicial or other adjudicative within the context of your audit objective(s),
system. may have occurred, consult with your
internal stakeholders, such as a legal expert,
and with SAI management to (1) determine
its effect on the audit findings; and (2) the
appropriate next steps to take based on
your SAI’s procedures.
How do you gather information using various methodologies?

There are numerous methods that audit teams can use to gather information. Still, all audit work
has to be conducted with the goal of obtaining sufficient and appropriate evidence to support
the findings of the audit. It is important that you ensure the audit you conduct will produce
evidence to support the development of findings and provide new information or analysis and
potentially support recommendations. There are multiple types of evidence, as discussed in
Figure 27.
125
Figure 27: Types of evidence
There are many different methods that audit teams can use to collect information and,
ultimately, produce evidence. This chapter will cover four common methods used for information
collection in detail:
• interviews;
• document collection;
• direct observations and inspection; and
• surveys.
The type of evidence that is most appropriate will vary depending on the audit questions and
how the evidence is used in the report (See GUID 3920/44-50). It is often beneficial to use
multiple types of evidence to support your findings and conclusions. Ultimately, it is important
to apply professional scepticism when collecting and analysing data, as the strength of your
evidence will rely on the reliability of the combined data in sum.
As you collect information, consider whether your audit work could provide insights related to
the economy, efficiency, and/or effectiveness of the audited entities. This means your audit work
could not just focus on what the audited entities did, but on how effective and efficient they were
in doing so and with what resources. It is also important to keep in mind the concept of
materiality as you determine what information to collect and how to collect it to better ensure
that your eventual findings will be of value. As discussed in Chapter 4, it is important to describe
in the audit plan the methods and information sources the audit team will use to gather evidence.
126
Depending on the complexity of the method, keep in mind that you may need to bring in
stakeholders, such as methodologists, subject matter experts, or consultants from inside or
outside your SAI to help you implement your chosen audit plan or provide advice as you conduct
audit work (GUID 3910/81). If you do not have access to experts that can assist you with complex
methods, then it is important for your audit team to select data collection methods that your
team has the training, competency, and resources to carry out (GUID 3910/79-80). Finally, it is
also recommended that you carefully consider the data that a method may yield and any
limitations before beginning data collection.
Interviews
Interviews are an important evidence-gathering tool for performance audits and will generally
be your primary means of gathering testimonial evidence. An interview is a question-and-answer
session that is designed to elicit specific information – and, in the case of a performance audit,
appropriate evidence. Interviews also provide a good opportunity for you to gain insights about
potential sources of documentary evidence. An auditor’s ability to interview effectively and then
accurately document the information provided during the interview will influence the quantity
and quality of the evidence collected. A well-designed and executed interview can yield:
• the perspective and observations of the person(s) being interviewed;
• documents and information or data provided by the person interviewed; and
• referrals to other people or offices for additional information.
There are two general types of interviews – unstructured and structured.
• Unstructured interviews are designed to elicit a full discussion of the interviewee’s

observations and knowledge about the interview topics. The questions are not prescribed,
and how you ask them is flexible and dependent on the interview. The responses are also not
defined – that is, the interviewee can answer the questions any way that they would like
instead of selecting from a list of potential answers. Examples of open-ended questions that
an auditor might ask during an unstructured interview include:
o Please briefly describe the state’s activities regarding the prevention of domestic violence
against women.
o What are the state’s main obstacles, if any, to correctly applying the laws protecting
women from domestic violence?
o Based on your experience, what can be done to improve the service for women victims of
domestic violence?
127
• Structured interviews are designed for an auditor to ask a prescribed set of questions
uniformly, usually offering a defined set of possible responses. It is recommended that you
consider your audit questions and the evidence you have already collected to develop
reasonable and likely response options for a structured interview. This approach is useful
when you want to quantify responses. That is when you want to say, “Of [the number of]
people we interviewed, [this number of people] said … .” It is often used when conducting
interviewer-administered surveys, such as telephone surveys. An example of a closed-ended
question that an auditor might use in a structured interview is below:
o Example: What problems, if any, do the police face in delivering services to women victims
of violence?
( ) Insufficient staff
( ) Lack of capacity to listen respectfully and without prejudice
( ) Lack of proper reception
( ) Few police officers with skills in gender issues
( ) Inadequate facilities
( ) Lack of standards
( ) Lack of information about women’s rights
( ) Other. Which? __________________________________________________
For example, the European Court of Auditors conducted an audit using both result-oriented and
system-oriented approaches to examine the degree to which the European Union’s (EU) efforts
to mitigate risk in the agricultural sector were efficiently implemented and were effectively
delivering results. As part of this review, the audit team conducted interviews with 105 farmers
in 17 different EU member states to discuss, among other things, the causes of production losses
for the farmers (for example, climate events, pests), the preventive measures taken at farm level
(for example, crop rotation, sanitary measures) and the degree to which farmers are insured
against the risk of loss. The interviews included structured questions, which allowed the audit
team to effectively quantify the responses. For more details about how this method was used to
support the audit team’s findings, see Special Report no 23/2019: Farmers’ income stabilisation:
comprehensive set of tools, but low uptake of instruments and overcompensation need to be
tackled.
An interview can also be semi-structured, meaning that your set of questions includes both
prescribed and flexible questions. The approach you choose will depend on how you want to use
the responses. The typical interview will likely include both open-ended and closed-ended
questions.
128
Tips for effective interview questions
✓ Ask objective, neutral questions without the ✓ Keep your questions simple, clear and
implication of bias. concise.
✓ If you seek an open-ended response, avoid ✓ Do not try to cover two issues in one question.
questions that can be answered with a ‘yes’
✓ Use probing questions to encourage further
or a ‘no’.
discussion about important topics without
✓ If you seek a closed-ended response, ask biasing responses. For example, “Could you
questions that restrict answers to a ‘yes’, ‘no’ tell me more about that...?” or “I am not sure
or other specific response. I fully understand the process. Could you
elaborate?”
To be effective, interviews must be planned well, conducted with care and skill, and documented
fully and accurately. Also, remember to consider people outside the audit organisation with
relevant and valid knowledge about it (for example, clients, civil society organisations, experts
and other government entities). There are generally three phases involved in carrying out
effective interviews – planning, conducting and documenting the results:
1. Planning the interview involves the necessary research, administrative and logistical
activities you need to conduct before you can effectively interview an official:
• Identify the office or individuals to be interviewed. If you are unsure, ask your primary
contact at the audited entities to identify these individuals.
• Plan the logistics for the interview, including working with the audited entities to schedule
the time and location of the appointment. Good practice is to have at least two members
of the audit team present at all interviews so that each member of the team can
corroborate the other members’ understanding of what was discussed.
• Conduct pre-interview research to ensure you are knowledgeable about the topic and the
role of the individual(s) you will be interviewing.
• Develop questions for the interview based on the information you need to elicit. If you
are interviewing an individual from the audited entity, make sure your questions include
enquiries about the degree to which the entity is achieving its objectives (effectiveness),
the resources it requires to carry out its mission (economy) and the relationship between
resources employed and outputs delivered (efficiency). If you have well-defined criteria
that are relevant to the interview topics, it may be useful to derive questions from these
criteria to make it easier to analyse the information later. Depending on the situation, you
may want to send these questions to the audited entities ahead of time so that they can
ensure the correct individuals are present and prepared to respond to your questions. It
is also useful to think about potential follow-up questions so that you are prepared to
probe the interviewee further during the interview as necessary.
129
2. Conducting the interview involves carrying out the planned interview to elicit the
information you need, including collecting related audit documentation and data:
• Determine who will lead the interview. It is common practice for one person to lead the
interview and the other members of the audit team to be responsible for taking notes.
• At the outset of the interview, provide introductions of the audit team and interviewees,
a statement of purpose for the interview and background information on the audit.
• When interviewing officials, ask relevant questions and take careful notes of their
responses. It is important that you ask follow-up and probing questions to improve the
quality and depth of your evidence. For example, a useful probing question is, “Can you
give me an example of that?” It is also important to probe for and evaluate any contrary
evidence that may exist to help you to understand the full picture and avoid incorrect
conclusions. Be prepared to adjust or go beyond your planned list of questions if other
issues relevant to the audit objective(s) are identified during the interview.
• Maintain control over the interview to keep the conversation focused on the topics of the
interview.
• Request related documentation and information to corroborate or expand upon the
testimonial information provided by the officials. Explain to the interviewees how the
information you are gathering is relevant and needed for the audit.
• At the close of the interview, summarise key information gathered and the documents or
data the individuals have agreed to provide to your audit team. Address any final
questions or comments from the interviewees, and thank them for their assistance. You
may also want to let the interviewees know that you may need to follow up with them as
the audit progresses.
130
Tips for conducting effective interviews
✓ Be prepared. Study the subject and ✓ Don´t talk too much – listen and observe.
understand the role of the individual(s) you
✓ Be flexible but have in mind the goal of the
are interviewing.
interview.
✓ Prepare a list of the questions to be asked
✓ Be brave enough to ask difficult questions if
during the interview in advance.
relevant to the audit; be frank and candid.
✓ Schedule the date, time, duration and
✓ Avoid asking complex questions,
location of the interview in advance.
demonstrating ego and displaying
✓ Bring more than one person from your audit excessive knowledge or attitudes of
team to the interview. superiority.
✓ Assign roles to each person before the ✓ In the case of evasive answers, use pauses
interview, such as who will ask the questions or silence to indicate that you are waiting
and who will take notes. Avoid doing for complete information.
interviews alone if possible.
✓ Take accurate and comprehensive notes.
✓ Start and end the interview on time.
✓ Consider bringing an audio recorder, if
✓ Be attentive, observant, objective, respectful, appropriate.
impartial, sensitive and confident.
✓ Document the interview as soon as
✓ Create a rapport with the interviewee: an possible after conducting it.
interview is not a cross-examination.
Source: Adapted from AFROSAI-E Performance Audit Template Manual, 2013; SAI Brazil – Interviews in audit
3. Documenting the results of the interview involves creating an accurate written record of the
information that was obtained during the interview in a way that facilitates analysis and
quality control (GUID 3920/100). See Appendix 8 for a template to document the interview:
• Be as accurate as possible. You will be editing, summarising and synthesising information
as you develop the interview record. Still, it is important that you ensure your paraphrases
and changes are true to the information provided.
• Organise the written record in a way that will help your team analyse the information
obtained. For example, you could organise the record by audit question or topic area and
use subject headings to draw attention to different areas.
• Document the names of the individuals you interviewed and their titles and contact
information. This is essential for maintaining an accurate record of the interview.
• Differentiate between the official position of the audited entity that the interviewee may
have provided and the interviewee’s opinion on a matter. This is a significant
consideration in determining the appropriateness of the information.
• It is useful to reference and electronically link the documents that were provided by the
interviewee in the interview record where relevant. This will help to clearly explain the
documentation in context with the interviewees’ statements.
131
• Take steps to verify and confirm the accuracy of the interview record. Some audit teams
share their interview notes with the individual drafting the interview record to ensure
they have a comprehensive set of notes from the meeting. Other audit teams have one
person draft the record based on their notes and then have the other team members
review it for accuracy based on their notes. You can choose the approach which works
best for your team, but it is important to ensure your teammates who attended the
interview review the record to confirm its accuracy. It is recommended that you follow up
with the interviewee if you are unsure or do not understand any of the information they
provided. In some instances, you may also be able to record and transcribe the interviews.
When appropriate, audio-taping the interview can make it easier for you to listen closely
to what the individuals are saying, as you will not need to concentrate on taking notes. If
you decide to record the interview, ask for the interviewee’s permission and keep in mind
that recording the interview might prevent the interviewee from speaking freely on
sensitive issues. It is recommended that you consult your organization’s policy on audio-
taping interviews because practices vary widely by SAI.
To obtain a comprehensive view of the audit topic, it is important to interview people with
different positions, perspectives and insights. Since the results of your interviews will be
testimonial evidence, conducting many interviews with different people or offices can help
increase the strength of your evidence. Conducting interviews is resource-intensive, though, so
limit your interviews to what is necessary. One way to determine this is to consider whether
conducting additional interviews will add relevant new or interesting information that you cannot
obtain from other sources, such as from documents. It is important to remember that the
reliability of testimonial evidence obtained through interviews is dependent on the person who
provides it and their level of knowledge or bias. It is recommended that you corroborate the
information obtained whenever possible with documentation or another form of evidence to
mitigate audit risk, as discussed in Chapter 4. See Appendix 8 for an interview guide that contains
more details about how to plan, conduct and document interviews.
132
Document collection
When evaluating
The typical audit will rely upon a wide range of documentary evidence if documentary
evidence is appropriate, these
to support its findings and conclusions. Thus, document collection is a questions are useful to consider:
very important method of obtaining evidence.
• Does the document represent
Documentary evidence is generally considered to be more reliable than the official position of the
audited entity?
testimonial evidence. It is important to have documentary evidence to • Is the document a draft or the
corroborate the testimonial evidence you obtain (see Figure 27 and final version?
GUID 3920/74-77). You can collect documents from many different • Is the document incomplete
or outdated?
sources. However, whether you can use the documentary evidence you • Was the document
collect as evidence depends on its authenticity and the integrity of the developed by the most
knowledgeable source?
sources and systems producing the information (see side bar). This is
• Does the source of the
discussed in more detail below. document have any biases
that could affect its reliability?
• Is the information accurate?
• Was the methodology used to
develop the document
sound?

Audited entities
For most audits, the audited entities are the primary source of relevant documentary evidence.
Be sure to request from the audited entities documents that provide evidence to answer your
audit questions. This documentation could be either qualitative or quantitative. Examples
include:
• policies, guidance and organisational charts;
• contracts, invoices, accounting information and budgetary data;
• quantitative data about the performance of the topic being audited; and
• research or studies related to the audit topic.
At the beginning of the audit, it is useful to ask the audited entities for documentation to provide
you with information about its organisation, operations and guidance related to the relevant
topic area. Collecting and reviewing this information early in the audit will help prepare to
effectively conduct interviews, surveys, additional document collection and inspections as the
audit progresses.
Remember to ask for documentation that substantiates officials’ statements, establishes relevant
facts and provides insights into how effective and efficient the audited entities are in performing
its role relevant to the audit objective(s) and questions.
As you collect documents from the audited entities, it is your responsibility to assess if the
information is appropriate. You cannot assume, just because a document or data was provided
133
by the audited entities, that it is relevant, valid and reliable. For example, the audited entities
may not have accurate information or have performed accurate analysis itself, or it may provide
you with information that presents a biased or incomplete view of the situation. Make sure you
understand how data and information was developed, and that information in the documents is
consistent with what you have been told by the audited entities. For example, you can ask the
same questions of multiple people about the origin of the information and collect similar types
of information from different sources to corroborate what is provided by the audited entities and
to ensure you have a complete picture. You may also want to ask to review the source data, cases
or files that underpin an audited entities’ analysis or conclusions so that you can verify the results
yourself. Also consider the timing of the documents that you are reviewing. Specifically, if you
are examining documents related to a specific event, determine whether the document was
prepared at or close to the time of the event. For example, were the meeting minutes prepared
the same day or six months later? This could affect the validity and appropriateness of the audit
evidence.
It is useful to maintain a register to record and control all documents you collect during the audit.
This will assist you in keeping track of the documents you have requested, what the audited entity
has provided, and what documents are still outstanding.
Depending on the audited entity and sensitivity of the topic, you may face challenges obtaining
documents or information from the audited entity. If an audited entity is trying to prevent you
from obtaining information that is relevant to your audit questions, it is recommended that you
notify your supervisor immediately so these issues can be quickly escalated and resolved in
accordance with your SAI’s policies and legal rights.
134
Challenges obtaining information from the audited entity and how to address them
Based on the laws of the country, each Supreme ✓ ensure your request for information has a
Audit Institution (SAI) has to have the legal right to direct relationship to specific audit questions;
access relevant government documents and ✓ explain the nature of the request to the
information to support the audits they undertake. audited entity as specifically as possible and
Developing a positive relationship with the audited link it to your specific audit question(s);
entity, including communicating frequently about ✓ set specific due dates for receiving
the information you need and why it is needed to requested information or meetings;
support the audit, can help you obtain information ✓ if the requested information is legitimately
more easily. sensitive, work with SAI management and the
audited entity to determine if there is an
However, some audited entities may not readily
alternative source of information that would
provide access to the information you request. If
meet the needs of the audit or if an
you are having difficulty obtaining information to
acceptable accommodation, such as
which you believe your SAI is legally entitled, such
reviewing the information on-site, can be
as through significant delays or denials to
reached; and
information:
✓ document the attempts you have made to
✓ notify your supervisor immediately so they are obtain the information and maintain a log of
aware of the issue and can escalate it to senior your requests.
SAI management, as appropriate;
✓ consult a legal expert within your SAI to ensure Your audit team will need to work closely with
the information you are requesting is SAI senior management to determine how to
information to which your SAI is entitled, and for resolve the issue.
advice on how to frame your request for the
information;
Third-party sources
Relevant third-party organisations – such as clients, experts, civil society organisations,
contractors, professional organisations, research organisations or other government entities –
which are not the primary subject of the audit, can also be useful sources for documentary
evidence. For example, a contractor may be able to provide you with information about its
performance relative to a contract. Or a research organisation may have conducted a relevant
study about the audit topic. As described in Chapter 4, it is always useful at the beginning of an
audit to conduct a literature search of general research reports, books or papers related to the
audit area to help you identify relevant sources.
Ensure that you understand the context, the third party’s role relevant to the topic and any
potential bias or motivations of the third party when considering whether the source is
appropriate to use as evidence.
Collecting information from a knowledgeable and relevant third party can be especially useful if
you doubt the trustworthiness or openness of the audited entity. In such circumstances,
information from a third party can help to either corroborate the information provided by the
audited entity or help you develop a complete picture of the audited activity.
135
File reviews
A well-defined data
File reviews involve reviewing many similar types of documentary collection instrument is
records, such as personnel files or contracts, to extract information. File important to a successful file
review. Ensure that you:
reviews need to be structured and systematic to allow for the issues or
questions to be addressed y across files. Similar to direct observation, it • understand the contents of the
files before developing your
is important that you identify the information you need to collect and data collection instrument;
develop a data collection instrument before beginning information • carefully develop the questions
that will help you capture the
collection. See Appendix 9 for an example of a comparison between two desired information from the
files. files; and
• test the data collection
instrument on a small number of
files to ensure it captures the
Web-based sources needed information.
Audit teams will often use web-based sources to obtain information.

Sources may include the websites of government agencies, legislative Source: IDI/PAS Development Team
bodies, trade associations or media outlets. Using information from certain websites is associated
with a higher risk that the information is not appropriate. For example, information from blogs,
wikis and personal websites is not recommended to be used as evidence because these sources
do not have any identifiable, recognisable authority, or their authenticity cannot be verified.
Other websites – such as those related to trade journals or newspapers – may be authentic but
not necessarily authoritative or reliable. Use professional judgement when using information
from these sites.
You will need to carefully consider whether the website you are using is a reliable source to use
for the specific information you are considering using from the site. Ask yourself these questions
about web-based sources:
• Is the source authentic?
• Is the source authoritative on this topic?
• Is the source reliable?
• Is the source unbiased?
If using information from web-based sources, it is also important that you report on what date
you retrieved the information because web-based information can change. Ultimately, using your
professional judgement and applying professional scepticism will be critical in deciding whether
to use web-based sources and the information derived from them.
136
Computer-processed data
Audit teams frequently obtain computer-processed data as a source of documentary evidence,
such as data extracts from databases or software applications, data maintained in spreadsheets,
data collected from forms and surveys on web portals.
As with any data source, you cannot assume the data are reliable. If the data are not reliable, you
cannot trust that the information is valid. If the data you obtain are expected to materially affect
findings, conclusions or recommendations, you will need to take a few additional steps to ensure
the data are complete and accurate. Completeness refers to the extent that the data records you
need are available and that data fields in such records are populated appropriately. Accuracy
refers to the extent that the recorded data reflects the source information.
There are some potential steps you can take to assess the reliability of your data source. The
extent of your assessment will depend on how significant the data are to your findings. Potential
steps include:
• interviews with knowledgeable officials about the data sources and how data are collected,
processed and validated;
• electronic or manual data testing for missing data, outliers or obvious errors;
• reviews of related internal controls, such as processes and procedures related to entering and
validating data; and
• a traced selection or random sample to or from source documents.
Some of these steps can be complex to implement. You may want to consider bringing in a
stakeholder, such as a methodologist or an auditor with previous knowledge of the topic, with
expertise in assessing data reliability for advice or assistance in determining what steps to take
and how to conduct the assessment.
It is recommended that you begin to assess the reliability of your computer-processed data as
soon as possible after identifying the data as potentially material evidence. See Appendix 10 and
Appendix 11 for a template for assessing data reliability and an example of data reliability
questions for the audited entities. Audit teams often analyse computer-processed data to
develop analytic evidence. It is recommended that you assess the reliability of the data before
conducting an extensive analysis of the data because analytic evidence is only as reliable as the
underlying data.
You will find that computer-processed data are rarely perfect. However, you will need to
determine if the data are sufficient for the specific ways you plan to use them. Considering the
risks of using the data is important, such as the sensitive or controversial nature of the data or
whether using the data might have a significant negative impact on the decisions of those who
read your audit report. It is also useful to consider the strength of your corroborating evidence,
137
as strong corroborating evidence could help to mitigate some of the risks of imperfect data.
Conversely, if your corroborating evidence is limited and you are relying heavily on the computer-
processed data as the sole basis for your findings, then the importance of its validity and
reliability is further amplified. The decisions you make about the reliability of computer-
processed data may require the collective professional judgement of your audit team,
management and data experts within your organisation.
Remember, you should only use computer-processed data if you determine that the data are
sufficiently reliable for the purposes for which you are using it. Also, when reporting computer-
processed data in your final audit report, it is recommended as a risk assurance step that you
disclose some methodological information about the data you obtained, how you obtained it and
any limitations of the data.
Direct observation and physical inspection

It is important that you get away from your desk and observe the people, activities, procedures,
property or events related to your audit. These methods of information collection are referred
to as direct observation and physical inspection. Evidence obtained through direct observation
and physical inspection is known as physical evidence. It is generally considered to be one of the
strongest forms of evidence and more reliable than indirect evidence – that is, evidence provided
to you by the audited entities or third party.
These methods can be very useful if your audit questions relate to the condition of items or
property, accounting for inventory or whether an operation is being conducted as intended.
Using these methods can help you understand the context of the issues related to the audit and
how the related areas are working.
For example, the European Court of Auditors conducted a result-oriented audit of animal welfare
in the EU. The audit team selected a sample of five EU member states based on the size of their
livestock sectors and the existence of weaknesses in their animal welfare compliance that had
already been identified. In each member state, the audit team conducted direct observations of
animal welfare inspections of farms, animal transport and animal slaughter. In addition, the audit
team conducted on-the-spot checks for farmers’ effective compliance with requirements
associated with their receiving payments and grants, such as whether animals have the legally
required grazing space and appropriate nutrition. For more information, see Special Report No
31/2018: Animal welfare in the EU: closing the gap between ambitious goals and practical
implementation.
Figure 28 provides some additional examples of audit topics that may benefit from direct
observation or physical inspection and related observations or inspections you could consider.
138
Figure 28: Examples of direct observations or physical inspections
Maintenance of Conduct site visits to relevant properties to physically inspect the buildings based
government- on criteria established in the contract. Take photographs and document the
owned facilities conditions you observe.
by a contractor
Procedures for Visit relevant airports and observe how customs inspections are being conducted.
customs Record your observations so that you can compare what you observe to the audit
inspections at entity’s procedures for conducting inspections. This may help you determine if
airports inspections are being conducted according to the specified procedures and the
level of resources that are required to conduct such inspections. You could also
consider during your observations whether there are ways for the audited entity to
be more efficient in the way they conduct their inspections.
Approvals for Inspect relevant files to check for the signatures and credentials of the approving
large purchases officials in accordance with legal requirements. You could also use this type of
of equipment information as part of a broader review to help you determine whether the
guidance and training for approving officials is sufficient to ensure they comply
with legal requirements or to determine whether the audited entity has sufficient
internal controls in place to ensure the law is followed.
Chemical Observe controls in place at border inspection sites to inspect food to determine
hazards in whether states effectively comply with food safety policies by conducting the
food appropriate physical checks of imported products of animal and non-animal
origins and with what resources.
Source: US GAO; European Court of Auditors Special report no 02/2019: Chemical hazards in our food: EU food safety policy protects us but faces challenges, 2019
Some direct observations are simple and may just require a few photographs or a video as you
are touring a warehouse or site. For example, you may interview an official about the damage
caused by flooding at a government site. You could then take photographs of the damage to
corroborate the official’s statements.
However, direct observations or physical inspections that are intended to directly answer or
partially answer your audit questions need to be conducted systematically. Consider talking to a
stakeholder with expertise in this area, such as a methodologist, for guidance or assistance in
implementing these methods.
Below are general steps to be taken to ensure the information you collect from your observations
and inspections are relevant, valid and reliable:
1. Determine what you will observe or inspect. Determine what sites, people, events or files
you will observe or inspect. If the universe is small, you may be able to conduct observations
or inspections at all or most of the sites or events. However, if you have a large potential
139
population to consider, you may have to select a sample of sites. If this is the case, it is
recommended that you talk to a methodologist to help you determine which sites or events
are best to observe or inspect to obtain the most appropriate evidence for your audit and how
those results can be used.
2. Determine what condition should exist. Determine the condition that ‘should’ exist – that is,
your criteria – before conducting your observations or inspections. The source of these criteria
will depend on your audit objective(s) and questions. Still, it could be determined through a
review of contracts, inventory records of the audited entities or required procedures. Chapter
4 discusses audit criteria in detail.
3. Determine what evidence you will collect and how. Based on the criteria you have
determined, develop a structured set of questions for you and your audit team to answer as
you conduct the observations or inspections. This may be referred to as a data collection
instrument. See Appendix 12 for a sample data collection instrument. This set of questions has
to be simple for you and the audit team to consistently answer at each observation or
inspection, even if conducted separately. The information you intend to collect can be
quantitative (for example, numbers of items) or qualitative (for example, descriptions of an
event or condition). Seek evidence that will help you evaluate the economy, efficiency and
effectiveness of the audit topic. For example, if you are observing how customs inspections
are being conducted, you do not want just to determine that they are being conducted. You
may also want to assess how quickly (efficiency) and thoroughly (effectiveness) they are being
conducted and with what resources (economy). In addition, determinations that you make
about how you will conduct your observation – such as conducting a covert vs. an overt
observation or observing a process as a participant – can affect the quality of the evidence.
For instance, customs officials who are aware that you are observing their inspections may
follow procedures more closely than those who are not aware.
4. Document the results. Carefully and accurately document the results of your observations or
inspections – that is, what exists – by answering the questions you have developed as you
conduct the inspection or observation (GUID 3920/100). Keep in mind when, where and how
the inspection or observation occurred and ensure it is recorded or documented in a way that
fairly represents the facts. For example, if an emergency event occurs during your observation,
the audited entities’ response to that event may not reflect typical operations for the entities.
It is also important that you record what you observe rather than your interpretation of what
you observed. Analysis of this information should come later. See Appendix 13 for a sample
template for documenting direct observations or physical inspections.
140
Conducting site visits
The typical audit requires many types of evidence and methods for collecting information. When
conducting an audit, you often may have less time, staff resources and money than desired. This, as well
as needing to use your SAI’s resources wisely, necessitates that you collect information in the most efficient
way possible. One technique that most auditors use to do this is by conducting a site visit that combines
multiple interviews, document collection and direct observations or physical inspections in a single visit to
a site or geographic location. Here is an example of how a site visit could be used to support an audit
related to the management of training for customs inspectors.
Sample site visit to assess the sufficiency of training for customs inspectors. For a system-oriented audit
question related to the management of training for customs inspectors, an audit team could
potentially conduct the following information collection in a multiple-day site visit to the city where the
training programme is located:
Day 1 Day 2 Day 3

Visit the academy that Return to the academy to Visit the local airport to
provides customs inspection observe training and to take observe inspectors
training to new inspectors to photographs or video of conducting customs
conduct interviews of the training and associated inspections and to interview
programme administrators, training aids. inspectors and supervisors.
the officials who develop the
training curriculum and the
officials who provide the
training.
Scheduling a comprehensive site visit will require planning, careful scheduling and an
understanding of how the audited entity or subjects of the visit are organised. However, the extra
effort to do so will allow you to collect far more evidence in a short period than if you conducted
interviews and physical observations on separate visits to the location.
Surveys
Surveys are another information collection method that audit teams can use to obtain evidence.
A survey is a systematic collection of information from a defined population that can provide you
with self-reported information about existing conditions or programmes. Surveys may be self-
administered by questionnaire (for example, mail, email or web surveys) or interviewer-
141
administered (for example, face-to-face or telephone surveys). A survey could be a useful method
to consider for your audit if you need to gather detailed and specific information from a
comprehensive group of people, offices within an organisation, or organisations, such as to
measure the level of satisfaction of a targeted user population with regard to public services
rendered.
For example, the United States’ (US) Government Accountability Office
If you plan to survey (GAO) conducted a review of early childhood education programmes
members of the public,
traditional or social media could provided by each of the 50 US states. The audit teams took a system-
provide you with effective
oriented approach and sought to determine the number and
options to reach your intended
audience. characteristics of these programmes, how they are funded and the degree
For example, US GAO recently
to which they overlap with federal and other state programmes. As part of
used social media to survey its review, the audit team conducted two surveys. Each survey was sent to
members of the US population
who have lived in privatised
early childhood education programme officials in each state. The first
military housing. survey identified state programmes providing early learning or childcare
If you use the media to contact
services to children in the 0-5 age group. The second survey gathered more
survey populations, take care to information about the programmes identified in the first survey, including
ensure you are using methods
that are inclusive – that is, their characteristics and funding sources. The audit team then analysed the
methods that will reach all survey data to determine which characteristics state programmes shared
subpopulations of your intended
audience – so that you do not with federal and other state programmes, as well as the benefits and
inadvertently bias the results. challenges of using multiple funding sources. For more details about these
surveys and the results, see Child Care and Early Education: Most States Offer
Preschool Programs and Rely on Multiple Funding Sources (GAO-19-375).
It is important to note that designing and administering a survey that produces objective, credible
and reliable information is a complex and time-consuming. A considerable amount of upfront
work is required to develop and test the survey. This work, and the time commitment it entails,
is often overlooked by audit teams when considering this method. Before embarking upon a
survey, ask yourself whether there are alternative sources of information available that could be
used effectively instead of the survey or as corroborating evidence with the testimonial
information collected from the survey.
Some of the key steps in administering a survey are briefly highlighted below and discussed in
more detail in Appendix 14. If your audit team is considering a survey, it is recommended that
you seek out a stakeholder within or outside your SAI with expertise in the design and
administration of surveys to provide guidance and assistance.
1. Identify the survey population. You need to identify the population you will survey, including
whether you will survey the entire population or a sample. In doing so, you have to ensure
142
that the individuals or organisations are the best sources of the information you seek. The
box below provides only a brief introduction to the concept of sampling, but there is much
more to learn about sampling and how it can be used. It is recommended that you seek the
advice of an expert and review academic literature when considering a sample.
Sampling
Sampling can be a powerful tool for estimating the characteristics of a population when you cannot
collect information on the whole population. A sample is a group of people, sites, objects, items, or
documents taken from a larger population for measurement. An audit team could use sampling as a
tool for multiple data collection methods, including document reviews, physical inspections, or surveys.
There are two general types of samples: probability and non-probability.
Probability sample Non-probability sample
A probability sample uses random sampling Non-probability samples are simpler but more
techniques to create a sample. Every member restrictive in what they will allow you to say. Such
of a population has a known and equal samples may use random or non-random processes,
chance of being selected for such a sample. like auditor judgement or convenience sampling.
Random processes, if possible, are preferable,
Well-designed probability samples allow though they will not allow you to generalise your
analysts to make statements about an entire results across the population in this type of sample.
population and measure the accuracy of
their estimates. Non-probability samples can be useful when you
need descriptive information about your sample or if
you are trying to establish the existence of an
attitude or error rather than prevalence. They are
not recommended as the sole support for findings
involving estimates of variables.
Source: US GAO
2. Select a method for administering the survey. There are multiple methods you can use to
administer a survey, including face-to-face or telephone interviews, web-based surveys,
paper surveys via mail, electronic surveys via email, or in-person self-administered paper
surveys. The method you choose will affect the response rate to your survey if the target
population cannot easily respond to the survey or if you do not have the staff resources to
administer it as planned.
143
Survey response rate
The survey response rate may affect how you Keep in mind how varying response rates from
can use the information provided in a survey – different geographic locations, offices or
for example, whether the responses can be demographic groups could lead to bias or error
generalised across the whole population or in the results of the survey.
whether the responses can be used only in a
more limited scope. There is no minimum threshold for an acceptable
response rate. You will likely need to work with a
If you do not receive enough responses to your subject matter expert to ensure that you have a
survey from certain subpopulations, there is a sufficient response rate in total and across
chance that your results could be biased. subpopulations for the intended use of the results
of your survey or to assess and adjust for
nonresponses.
Source: US GAO
3. Analysing the survey responses. You will need to analyse the information obtained from the
survey to use it as evidence. The type of analysis required will be dependent on the types of
questions you asked and how you want to use the information. Potential techniques for
analysing evidence is covered in more detail later in this chapter.
4. Documenting the survey results. You will need to carefully document how you conducted
the survey, the survey responses and any analysis performed on the survey results.
Conducting an effective survey will require far more guidance than this handbook provides.
Remember to seek out assistance from a methodological expert, either internal or external to
your SAI, before attempting to conduct a survey.
Tips for conducting effective surveys

✓ Write clear, concise, accurate and neutral ✓ Only ask questions that will be used for
questions. analysis.
✓ Do not cover two issues in one question. ✓ Start the questionnaire with easy questions.
✓ Avoid ambiguous or vague questions. ✓ Avoid too many open-ended questions.
✓ Conduct pre-tests of the survey questions
with members of the target population.
144
Other potential methods for collecting information Site selection for case
studies has a direct
There are many methods you can use to collect information besides those impact on the data you will be
able to collect and your
this chapter has covered. Below are two additional methods that are more resulting findings. Ensure that:
commonly used.
• case study selections are
well-thought-out, defensible
Case studies and documented; and
• you choose sites that contain

Case studies are an in-depth, detailed examination of one or more complex a range of the characteristics
of interest. For example,
events, incidents or locations. You could use this approach to examine choose sites from rural and
processes over time, as well as the relationships between processes and urban areas, large and small
cities, or areas with lots of
outcomes. The goal of a case study is often used to answer complex ‘why’ activity and areas with little.
or ‘how’ questions. Case studies are time-intensive and often involve Source: IDI/PAS Development Team
multiple methods of data collection and sources of information. Because case studies are focused
on a single or limited event, the information obtained will not represent all events. In fact, case
study subjects are often purposefully selected because they provide particular or unique
perspectives. You can avoid bias in such cases by including subjects that offer multiple
perspectives and describing the differences objectively. Information obtained from case studies
works well in combination with or supplementing other data collection methods.
Focus groups
Focus groups are moderated discussions with groups of participants to

explore concepts or obtain information about their experiences related to
The goal of a focus
the topic (for example, the perspectives of customs inspectors regarding group is to have a robust
the quality of their training). A focus group is different from a group discussion among the
participants. To encourage
interview because it also aims at observing and exploring the interaction this:
among the participants. As with many data collection methods, the
• keep groups small – about
individuals chosen to participate could affect the appropriateness of the eight to ten participants;
information you obtain, so choose carefully to avoid biasing the results. • assure participants of their
anonymity; and
• create homogenous groups
so that participants feel free to
express their perspectives
honestly (for example,
managers with managers and
employees with employees).
145
Important factors to keep in mind while gathering information
✓ For most audit topics, there will be far more discrepancies to ensure that the evidence you
information available than you can gather use to develop your findings is relevant, valid
and analyse. It is important to set realistic and reliable.
expectations about the information that is
✓ Remember that it is your responsibility to
needed and can be collected during the time
exercise professional judgement and
frames of the audit.
scepticism and consider issues from different
✓ As you collect data, you may find perspectives. This will require you to maintain
discrepancies or disagreement between an open and objective attitude to various
information obtained from the various sources. views and arguments.
It is your responsibility to resolve these
How do you analyse information?
The Standard
The auditor shall analyse the collected information and ensure that the audit findings are put in
perspective and respond to the audit objective(s) and audit questions, reformulating the audit
objective(s) and audit questions as needed.
You will need to perform analysis of the information you have collected to understand and
explain what you found and ultimately to produce evidence. The goal of analysis is to use the
information collected to assess economy, efficiency and/or effectiveness and to answer your
audit questions. Focusing on the audit questions will help you organise your information and
ensure that your analysis will help you get the answers you need.
As discussed earlier in this chapter, information collection and analysis are often conducted
concurrently during the audit. Continuous analysis of your information throughout the audit will
help you identify if you are collecting enough of the right information to answer your audit
questions. This is part of your responsibility and enables you to actively manage audit risk and
avoid the development of incorrect or incomplete audit findings, conclusions and
recommendations or provide unbalanced information.
There are many different types of analytical methods you can use to analyse the information
collected. The methods you choose will depend on your audit questions and the nature of the
information (GUID 3920/86). Some common qualitative and quantitative methods of analysing
information and data are briefly discussed below.
146
What are key qualitative methods of analysing information?
Qualitative analysis includes a wide range of methods for structuring, comparing, compiling and
describing information that supports logical reasoning and arguments related to the evidence.
You would typically conduct qualitative analysis of evidence from interviews, documents and
surveys.
Specifically, you will have conducted many interviews and collected many documents throughout
your audit that contains evidence to help you answer your audit questions. Your audit questions
may provide a basic structure for analysing the qualitative information you have collected to
identify key evidence. Beyond this, there are many different qualitative approaches you can use
to analyse the documents, ranging from simple to complex methods that require planning. Figure
29 provides some examples of common methods of qualitative analysis that can be used in
analysing information from interviews or documents.
Figure 29: Examples of common types of qualitative analysis
Direct This type of analysis involves extracting information directly from documents or
interviews provided, such as information about the entity’s official plans and
actions or information related to the performance of the audited topic. This is
the simplest type of qualitative analysis, but it is important for you to corroborate
this information with other evidence you obtain.
Topical This type of analysis involves reviewing documents or interviews with a focus on
topical information that is relevant to your different audit questions. Searching for
common themes, similarities or differences can be useful in the development of
audit findings.
Chronological This type of analysis involves reviewing documents or interviews with the purpose
of establishing the order in which a series of events took place or to establish the
steps of a process.
Thematic This type of analysis involves identifying and counting the frequency of certain
expressions or themes in documents or interviews; for example, how often
summaries from management meetings include discussions on how to provide
more developmental opportunities for employees. This type of analysis will
require you to develop a clear methodology before you begin, including clearly
defining what will be counted and how.
Content This type of analysis involves structuring and analysing complex qualitative data
with the intent of distilling it into quantitative information. This is one of the most
complex types of qualitative analysis and will require you to develop a clear
methodology before you begin. See below for more information on how to
effectively implement this type of analysis.
Source: US GAO
147
Analysing documents
When analysing the documents that you have collected, the qualitative method(s) you use and
the complexity of the analysis required will depend on your audit objective(s), questions, and the
types of documents or other sources of information that you have. For example, if your audit
questions are related to the customs inspections requirements the audited entity has established
in agency guidance, and the audited entity has only one related guidance document. You may be
able to extract information directly from that one document – a method referred to as direct
analysis. However, if the audited entity’s requirements for customs inspections are contained in
10 different guidance documents, your analysis will need to be more complex to systematically
account for the guidance in all the documents. The more complex methods of qualitative analysis
discussed in Figure 29 above, such as content analysis, often require careful planning and clear
methodologies to effectively implement. See Appendix 15 for more information about content
analysis and an example.
Analysing interviews
The interviews you have conducted will also likely comprise a significant amount of your
evidence. You will need to select an approach to analyse your interviews to identify common
threads of information or topics, things that fit together, or examples of the same underlying
problem, issue or concept. For example, if one of your audit questions is related to the
effectiveness of training for customs inspectors, you could conduct a topical analysis by reviewing
each of your interview records and extracting all the information pertaining to the effectiveness
of training for analysis. The box below describes some simple steps of how such a topical analysis
based around your audit questions could be carried out.
148
How do you analyse interviews based on the audit questions?
1. Choose a method for structuring the 4. Compile and analyse the answers of each
information from the interviews, using audit type of key player, one at a time.
questions as the first choice; and sub- 5. Compile and analyse the answers of all types
questions, actors, regions, etc, as the next of key players together.
choice if it is not meaningful to structure the
6. Look for similarities and differences between
information only in line with the audit
the answers of different categories of key
questions.
players.
2. Read the interview notes again and focus on
7. Summarise the information and judge how the
the structure. If interviews are to be organised
interviews can contribute to answering the
according to audit questions, make a note in
audit questions and developing
the margin when something is relevant for
recommendations.
question number one, two, etc.
8. Continue with the next audit question.
3. Go through all the notes regarding audit
question number one. If there are many
relevant remarks, make a written summary. If
necessary, choose a new factor to structure
the remarks. Key players could be used as
such a structuring factor.
It is important to document what you find as you analyse the interviews. One common approach
is to develop a summary document to compile the information from the interviews related to
each audit question or factor. See Appendix 16 for a document summary example. At a basic
level, this involves grouping and labelling similar evidence in a way that makes it easy for you to
understand and evaluate. Having all the information organised and documented in one place will
help you understand the totality of the relevant evidence related to the topic. If you develop a
summary document, include the source information of each piece of the evidence – such as a
link back to the original interview documentation – to ensure the evidence trail is clear. Your SAI
may have access to software programs you can also use for this type of data analysis. This is
discussed in more detail below.
You have a unique opportunity to compile data from many different sources and listen to the
knowledge and views of many different members of staff on many levels within the audited
entities and third parties. As noted earlier, keep in mind as you are analysing the interviews that
the individuals whom you interviewed may have different perspectives on the issues and only a
partial view of the facts or the causes of a problem. It is your job as an auditor to evaluate all the
information provided to you in the interviews to come up with a more objective and
comprehensive picture of the performance of the audited entities.
149
What are key quantitative methods of analysing data?
Quantitative analysis ranges from simple (for example, calculating an average) to complex (for
example, statistical modelling) methods. In performance auditing, quantitative analysis can help
you uncover important patterns and relationships in your data and identify areas that need
attention or improvement. This section will briefly describe the types of quantitative analysis you
may want to consider in your performance audits.
Statistical analysis
Statistical analysis is the science of uncovering patterns and trends in data. It can range from
simple descriptive statistics to complex analysis like regression analysis (see below) that requires
sophisticated techniques and software.
Descriptive statistics
In performance audits, you will most often use descriptive statistics to help you understand,
summarise and describe distributions in the data you have collected in a meaningful way, such
as in analysing the audited entities’ achievement of performance targets by site or income
distribution in a population. Figure 30 describes some basic concepts in descriptive statistics and
when they can be useful.
150
Figure 30: Basic concepts in descriptive statistics
Concept Definition When to use
Mean The sum of a set of values divided by Useful when data points are
the number of values; also known as symmetrically distributed. Use caution if
average. you have data points that are extreme
outliers – that is, unusual when
compared to the rest of your data.
Median The middle value when the values are Useful when extreme scores or outliers
arranged in order of size; the 50th may distort the mean.
percentile.
Mode The most frequent value of a set of Useful when you are looking for the
values. most common category, popular
option or typical value.
Range The difference between the highest Useful to complement the mean and
and the lowest observation. median to discuss how data points are
distributed.
Variance Quantifies the extent to which Useful to complement the mean as a

elements of a population are spread measurement on how scores are
out from each other; average of the distributed.
squared distance between the single
observation and the mean value.
Standard Measure of the dispersion or spread in Useful to complement the mean as a

deviation the data; the square-root of the measurement of how data points are
variance. distributed; use caution if the data have
significant outliers.
Percentage A measure of a part or proportion Useful to understand the size of a part of

relative to the whole, expressed in population relative to the whole, such
hundredths. as the number of ‘yes’ answers in
relationship to the total number of
responses on a survey.
Index Measure of changes in a Useful to compare the development of

representative group of individual data variables over several years, or to
points; a compound measure that compare different years, such as an
aggregates multiple indicators. inflation index.
Source: Adapted from AFROSAI-E Performance Audit Handbook 2016:119
You may need to use multiple descriptive statistics to present a full picture of your data set
because a single figure – like the mean – may be misleading if there are outliers in the data set.
Figure 31 shows how some of these descriptive statistics could be used to describe the incomes
of staff at a factory.
151
Figure 31: Incomes of staff at a factory
Staff 1 2 3 4 5 6 7 8 9 10 11
Salary $12k $14k $15k $15k $15k $16k $18k $20k $22k $70k $95k
Mean: $28k
Median: $16k
Mode: $15k
Note: ‘k’ equals a thousand.
If you are asked to report on the typical salary at this factory, using only the mean could provide
a skewed view because of the two workers who have large salaries. The median and mode, in
this case, provide better measures of the typical salary of the workers at the factory. Providing
the percentage of workers in your data set who make less than a certain value could also be
useful in describing this data set. For example, nearly 82% of the workers earn a salary of less
than $25k.
Some of these concepts – variance and standard deviation, for example – can at times be
challenging to calculate and interpret. Software spreadsheet programs can assist with the
calculation. Though if you do not have experience applying these concepts in a performance
audit, it is recommended that you talk to an internal stakeholder with subject matter expertise if
you think such analysis would benefit you in answering your audit questions.
Regression analysis
Regression analysis is a statistical technique for assessing the degree to which variables are
associated with one another (for example, correlated).
Regression analysis can be useful in performance auditing if you are trying to:
• test a relationship that is supposed to hold true;
• identify relationships among variables that may be causally related, which can help explain
outcomes;
• identify unusual cases that stand out among expected values; or
• make predictions about values.
For example, the US GAO conducted an audit in 2018 that examined factors that affect university
preparatory course offerings at high schools in the US. The audit team took a problem-oriented
approach that began with the premise that poverty can adversely affect academic and other
outcomes in many ways. The audit team examined how high school students of different poverty
levels are offered courses to prepare them academically for college. To do this, GAO developed
152
a regression model to test the relationship between the offerings of university preparatory
courses and school characteristics, including poverty levels of students, school size, population
density of the area (that is, rural versus urban) and ethnic make-up of the student population.
Among other things, GAO’s regression analysis showed that schools with high poverty rates
among their students were less likely to offer the mathematics and science courses that most
public four-year universities expect students to take in high school. For a more detailed
explanation of this example and the audit team’s methodology, see K-12 EDUCATION: Public High
Schools with More Students in Poverty and Smaller Schools Provide Fewer Academic Offerings to
Prepare for College (GAO-19-8).
Appendix 17 includes a very simple application of regression analysis to illustrate its potential
usage. As with all types of modelling, regression analysis can be complicated and may require
specialised software for certain data sets or complex analyses with many variables. If you do not
have experience with this type of analysis, seeking out training, academic literature, or guidance
from a methodologist or subject matter expert can help you appropriately interpret and describe
the results of regression in your audit.
Trend analysis
Trend analysis is useful if you are looking for patterns or changes in your quantitative data. At its
simplest, trend analysis involves collecting data from multiple time periods, plotting that data on
a graph so that you can see how the data has changed and then determining the factors that led
to the change.
In performance auditing, trend analysis is frequently used to look at changes in budgets, costs
and programme performance. It may also help you examine the effect of a change in the
environment – such as a new law, programme or resource – on a specific variable.
For example, an SAI was examining the number of road accidents in different regions. This was a
problem-oriented approach in that the preliminary problem of road accidents was known, but
the causes and mitigations were not known. One region in the study – Region B – implemented
a programme to conduct risk-based traffic inspections, while Region A did not implement such a
programme. The auditors analysed the number of road accidents before and after the inspection
programme was put in place in 2007, as seen in Figure 32.
153
Figure 32: Road accidents in regions with different types of inspections
Number of road accidents
Region A
without any
programme
Region B
Start of a programme for

risk-based inspections in
Region B
Year
As you can see from this analysis, the number of road accidents began to change in Region B
shortly after the inspection programme was implemented. A few years later, the number of
accidents even began to decrease in Region B. The rate of increase in the accidents in Region A
also slowed down, despite having no programme for risk-based inspections.
While compelling, the data analysis alone did not tell the whole story. To complete their trend
analysis, the auditors had to do further investigation and analysis to determine whether there
were other factors that could explain the differences in road accidents in Regions A and B and
the decrease in accidents in Region B. For instance, in their investigation, they found that a
national campaign on road safety was launched around the same time as the inspection
programme in Region B. Thus, this was a contributing factor that the audit team had to consider
when determining the effect of the inspection programme on road accidents. This is also a good
example of how an audit team could use trend analysis to focus on questions of efficiency and
effectiveness – that is, what inputs were required to achieve the desired outcomes.
As with this example, determining ‘how has X changed?’ is often just the starting point in a trend
analysis for further examination to understand ‘why did X change?’. For this reason, make sure
that any findings and conclusions that you develop based on trend analysis consider the many
factors that could be contributing to the observed trends in the data.
You can learn more about the data collection and analytical methods discussed in this chapter —
and others — by reviewing academic or evaluation literature.
154
Using software for data analysis
A wide variety of commercial software applications are available that can assist you in conducting
both qualitative and quantitative data analyses. These applications range from commonly used
word processing and spreadsheet programs to more expensive and complex systems. For
example, you can use software programs to manage, organise and analyse large amounts of
qualitative data, including conducting content analyses. There are also many software programs
available that support analysis of large sets of quantitative data, advanced statistics and
modelling.
The use of these sophisticated tools can enhance your audit work and analyse much larger sets
of data than you can manage and conduct manually. Remember that the quality of the data is a
critical consideration when using such software programs. Software programs can only produce
reliable results if the underlying data are reliable.
Check with your internal methodologists and subject matter experts to find out what software
applications your SAI has access to that may support your work. Many companies also provide
open versions or trial versions of their software programs for free; this may be a useful option
for your audit team to consider if your SAI does not have a paid licence for a program you wish
to use.
Using graphics to analyse and visualise data
Using graphics to analyse or visualise data is commonly referred to as data visualisation. Simply
put, data visualisation is the presentation of data in a picture or a graphic to visually communicate
a quantitative message to help with analysis. Its goal is to enable auditors, as well as decision-
makers, to grasp difficult concepts and identify new patterns.
Data visualisation in its most simple form includes basic graphs and charts, such as the trend
analysis and scatter diagram shown in the examples above. In its more complex forms, it can
include the visualisation of millions of lines of data using sophisticated software.
If you have quantitative data, consider using data visualisation as an analytical method. Creating
charts of that data can enable you to more quickly and easily see the connections between data
points, make comparisons and understand causality than reading lines of text and numbers.
Figure 33 shows examples of the types of charts you can use in your analysis to display the same
information.
155
Figure 33: Examples of charts that can be used for data visualisation
Total budget for fiscal years 2016-2019

Ministry
Defence
Education
Health
Treasury
Common commercial software applications have capabilities that can assist you in creating
different visualisations. Still, your SAI may also have specialised software that can assist you in
visualising large data sets. Talk to an internal stakeholder with subject matter expertise to
determine what resources are available.
It is important to remember that data visualisation must be easy to understand for the reader to
be effective. The best graphics are self-explanatory, though in some cases, you may need to
provide the reader with some background information in table notes to give the information
appropriate context. Graphics are also intended to be complementary to the text of the report
and not repetitive – meaning that you do not need to repeat in the text the information that the
graphic provides.
Once you have completed your analysis and developed your findings, data visualisation can also
be extremely valuable for communicating the results of your audit. The United Kingdom’s
National Audit Office and the US GAO have created websites to share the interactive data sets
they have recently produced. Check out these links for some examples of how you can use data
visualisation in your audit reports:
www.nao.org.uk/search/publication_type/data-visualisations/
www.flickr.com/photos/usgao/
And check out this blog post on why you may want to do so:
www.nao.org.uk/naoblog/visualising-data/
How do you document and safeguard information?

As you collect and analyse your information, it is important to document or show your work in a
timely fashion and to safeguard the documented information. As discussed in Chapter 2, it is
important your audit team creates and uses a cross-reference system that establishes
156
understandable and transparent links between the documentation obtained during an audit. A
documentation system should: provide you with easy access to the information; enable
supervisors to review the work as part of their quality control procedures throughout the audit
(and reflect this review after it is conducted), and facilitate internal or external quality assurance
reviews.
As mentioned in the sections above, be sure to document what you are doing to collect the
information, how you are analysing the information and the results of your analysis. It is helpful
to do this while you are taking these steps so that the process is fresh in your mind and you can
recollect all the pertinent details. You must establish adequate documentation to provide a clear
understanding of the audit work that you carried out. In practice, this means that your
documentation should enable an experienced auditor with no prior knowledge of the audit to
understand the nature, timing, scope and results of the audit work that you performed and the
audit evidence that you obtained to support the audit findings, conclusions and
recommendations, and the reasoning behind all significant matters that required you to exercise
professional judgement (ISSAI 3000/87). Prompt supervisory review of your audit
documentation will also ensure that individual documents are complete, accurate, clear and
understandable. This is an important risk assurance step because it can also alert supervisors to
any problems with the audit (such as insufficient evidence or insufficient documentation of
information gathered that weakens its usefulness as evidence). (GUID 3910/82-84)
It is helpful to group your collected information and analyses, either electronically or paper-
based, by establishing an understandable folder system.
Protecting personal or sensitive information
Throughout the audit, you may collect personal (such as personally identifiable information) or
sensitive information from the audited entities. If this type of information is collected, you must
ensure it is adequately safeguarded. When you think you may begin collecting this type of
information or if you have begun to collect it, it is suggested that you contact the audited entities
to discuss whether and how you can report on this information and ascertain that you are
safeguarding the information in a manner that meets the audited entities’ and your SAI’s
standards. For example, sensitive information could include personally identifiable information
about an individual, such as a national identification number or a birth date. In another example,
certain information may be classified or otherwise prohibited from general disclosure by law or
regulations. In such circumstances, you may need to publish a separate, classified or limited-use
report containing such information and distribute the report only to those authorised by
legislation or regulation to receive it.
157
When conducting a performance audit, remember to...
… continue to assess and manage risk, and … continuously apply professional scepticism as
ensure the quality of the audit work, you collect information through consideration
through analysis of the evidence for of the credibility of the individuals whom you
sufficiency and appropriateness; interview and the data you collect – probe
communication with internal and external for and evaluate contrary evidence, do not
stakeholders; developing detailed audit take things at face value;
documentation, and supervision of the
… focus your information collection and analysis
audit work;
on the economy, efficiency and/or
… continue to assess the independence of effectiveness of the audited entity relative to
the audit team to ensure that you avoid the audit objective(s) and questions;
bias, or the appearance of bias that could
… ensure that evidence and other audit
cause others to call into question the
documentation is sufficiently complete and
impartiality of the audit team;
detailed to establish the work performed and
… frequently communicate with the audited evidence obtained to support significant
entity to collect data, ensure analyses are judgements;
comprehensive and verify that the factual
… consider the materiality of the information
basis for the findings are accurate and fair;
you are collecting and potential results of the
… communicate with internal, and as analyses you are conducting and apply
appropriate, external subject matter professional judgement to ensure that your
experts and stakeholders to get advice, audit work is focused on significant activities
support or alternative perspectives in of the audited entity; and
collecting information and conducting
… ensure that information is collected
analysis to enhance the quality of the
specifically from vulnerable populations so
works;
that data is inclusive of all affected parties.
158
Chapter 6
How do you develop findings, conclusions and recommendations?

Developing findings, conclusions,
and recommendations
• Identify findings of the audit. • Draft conclusions and recommendations, if
• Develop the message, with appropriate applicable.
balance on positive and negative findings.
The purpose of developing audit findings is to compare the audit criteria to your condition,
determine cause and effect (if relevant), assess your evidence, ensure your findings are based on
sufficient and appropriate evidence and develop conclusions and recommendations (if
applicable).

• What is an audit finding?
• How do you compare audit criteria to condition?
• How do you determine cause and effect?
• How do you assess your evidence?
• How do you develop conclusions and recommendations?
• How do you prepare for drafting your report?
Developing your audit findings can occur simultaneously while you are collecting your evidence
or sequentially after you have collected it. It can be helpful to begin to identify the elements of
potential findings while you are still conducting audit work because this can help you identify any
gaps in your evidence and the need for additional audit work.
159
What is an audit finding?
The Standard
The auditor shall analyse the collected information and ensure that the audit findings are put in
perspective and respond to the audit objective(s) and audit questions, reformulating the audit
objective(s) and audit questions as needed.
Once you have collected and analysed your evidence, it is important to turn your attention to
assessing the evidence to develop audit findings. According to GUID 3920/79, the audit finding is
‘what is’ compared to ‘what should be’.
Throughout the audit, the analytic process involves continuously analysing and assessing the
evidence and how it relates to the audit questions. This creative, iterative and collaborative
analytic process will help your team develop quality audit findings. Some audits address different
thematically-related issues, where the full story on each issue can be presented as one finding.
In these cases, an audit finding can be described as containing four elements, as shown in Figure
34.
Figure 34: Elements of a finding
What should be?
Criteria
What are the What is?

consequences? Effect Condition
Cause
Why is there a deviation from
the criteria?
160
However, when different findings are linked to each other, the full story may be presented in the
audit report as a whole, as opposed to individual findings. In such cases, it is the report as a whole
that needs to cover the four elements.
It is important to consider these four elements throughout your audit. They
In developing a finding,
can provide a framework that helps inform how much evidence needs to be
you need to ensure the
finding: (1) is consistent with the collected and how it can be analysed. Also, once you begin assessing your
evidence on which it is based;
and (2) answers the audit
evidence, you will need to determine what information is most pertinent to
question. your audit questions and how the separate pieces of information relate to
Your audit questions can also
each other. This evidence assessment helps you determine what the
help organise the information evidence means. It is important that teams consider and refine potential
you have collected, and your
analysis of that information can
audit findings, as needed throughout this process.
help you determine what it all
means.
The nature and significance of a Audit findings have to be constructed using a clear and logical framework
finding will often determine the that will allow for your supervisor, management and
type of evidence needed. The Balance is important in
more significant a finding is, the stakeholders to easily understand the audit criteria developing the audit
stronger the evidence that is applied as well as the conditions and the analysis of findings. Ask yourself these
needed to support it. questions to ensure you are
the nature, significance and causes of the situation providing a fair and balanced
Source: IDI/PAS Development Team found. Do not forget to consider your findings in the picture:
context of economy, efficiency and/or effectiveness, • What would a reasonable
as this can provide a way to demonstrate the need for corrective action. person expect the audited entity
to be able to achieve?
Your findings should also be objective and fair.
• What is the audited entity
To ensure the audit report is complete, it is important to include both good doing well relative to the audit
questions?
and bad points and give credit where it is due. This is because findings
should be placed in context: assessing an audited entity’s activities or • What positive actions has the
audited entity taken to address
programmes will usually mean that some things work well. An objective any negative circumstances
and fair assessment must reflect this totality and not solely focus on found through the audit?
deficiencies. Source: IDI/PAS Development Team
It is also important to consider materiality and apply professional judgement throughout this
process (these elements are discussed in greater detail in Chapter 2). As stated in International
Standards of Supreme Audit Institutions (ISSAI) 3910/112, findings are considered material if
they, individually or in the aggregate, could reasonably be expected to influence relevant
decisions taken by intended users on the basis of the auditor’s report. The auditor’s consideration
of materiality is a matter of professional judgement and is affected by the auditor’s perception
of the common information needs of the intended users.
161
Example of an audit finding
Finding statement. There is a shortage of psychiatric inpatient beds in most of country X’s regional areas.
Criteria. The number of needed psychiatric inpatient beds established by the World Health Organization
is 0.43 per thousand inhabitants.
Condition:
• The country has an average of 0.37 psychiatric inpatient beds per thousand inhabitants.
• Uneven distribution of beds between geographic regions (the south eastern region has 0.53 beds per
thousand inhabitants, while in the northern region, the rate is 0.04) means their number of beds does
not meet the World Health Organization’s population criteria.
Causes. Country X did not consider how many beds it was distributing in each geographic region
because it does not have municipal and state mental health plans.
Effects:
• Deficiency of service in places with low bed rates.
• Migration of people with mental disorders among municipalities or states, complicating the planning
of healthcare.
How do you compare audit criteria to condition?
The Standard
The auditor shall identify the audit criteria and their sources in the audit report.
The backbone or core of your audit findings is the criteria and the condition. Condition is the
situation found, the most relevant occurrences identified in the fieldwork. To develop findings,
you will need to:
1. review the totality of information collected during your audit;
2. decide which items are most important to answering the audit questions; and
3. determine how the items logically relate to each other.
This evidence assessment process consists of combining information from the different data
sources to gain information and knowledge about the actual conditions. This means that:
information from interviews may be combined with analysis of statistical records; information
from case studies may be combined with information from surveys, and some information may
come from field studies in one province while other information refers to another province.
Combining this information is like completing a jigsaw puzzle, where the pieces are the different
162
elements of information and analysis. Assessing your evidence allows you to compare your
criteria to the factual situation or condition.
If there is a deviation between the criteria and the condition, then an audit finding that could
lead to a recommendation is generated. For example, if your evidence assessment shows the
audited entity or entities are not meeting the criteria, this could indicate an area where
improvement is needed. It is important to base the comparison of the criteria to your condition
on what a reasonable person would expect, considering the audited entities’ circumstances.
If there is no discrepancy between the condition and the criteria, then the audited entity has
done what was expected based on the criteria. If your assessment of the criteria and the
condition shows the audited entity is meeting or exceeded the criteria, then that could
potentially indicate a positive finding. It is important to include positive findings in your report
when your evidence supports them.
However, if a deviation between the criteria and the condition is identified, or the audited entity
is not acting consistent with the criteria to which you assess them. The next step after this
assessment is to analyse and confirm causes – that is, why there is a deviation from the criteria.
This could lead to a potential recommendation. For example, for an audit question related to the
sufficiency of training for customs inspectors, if you find that customs inspection training given
to new inspectors does not meet the training curriculum guidance, this could indicate an area
where the audited entity needs to improve.
Sometimes, the lack of information about your audit objective(s) or questions can be a finding in
itself. For the same audit question related to sufficient training for customs inspectors, you may
find that the audited entity does not collect information about whether the customs inspectors
that took the training believe the training prepared them for their jobs. This could then indicate
that the audited entity may need to collect this information so that it can make more informed
decisions about the training.
How do you determine cause and effect?

Ideally, you will have sufficient and appropriate evidence for determining cause and effect (or
consequences). To some extent, you can also use evidence on performance problems as a
springboard for determining both cause and effect. The cause is the factor or factors responsible
for the difference between the condition and the criteria and may also serve as a basis for
recommendations for corrective actions. Common factors include poorly-designed policies,
163
procedures or criteria; inconsistent, incomplete or incorrect implementation; or factors beyond
the control of programme management. It is important to note that establishing cause and effect
does not necessarily imply causation. It will be necessary to use enhanced analytical techniques
to answer questions on cause and effect. Because determining cause and effect is very
challenging from a methodological standpoint, it is important to consult with a methodological
expert or statistician during this process.
Correctly identifying the cause will sometimes require you to develop a causal ‘chain’ – that is,
moving further and further backward in your analysis until you can identify the specific thing that
most needs to be fixed. For example, ascribing poor evidence to inadequate planning may be
insufficient. What was the reason for inadequate planning? Was it misplaced priorities?
Something else? If you do not believe the cause is reasonable or credible, you may want to
explain your concerns to the audited entities and hold further discussions. Frequently asking the
question ‘Why?’ during data collection can enable you to identify and analyse causes for
identified performance problems.
You can determine the effect by comparing the actual condition to the ideal situation, had the
criteria been met. You can identify effect as either what has already occurred or a likely future
impact based on logical reasoning. You can also identify positive effects (by doing this action, the
audited entities will be able to achieve a particular economy, efficiency and effective outcome)
or negative effects (without doing this action, the audited entities will not be able to achieve a
particular economy, efficiency and effective outcome). Do not forget that other external factors
can also influence the observed effect.
It is also important to understand the nature of any relationships that may exist between cause
and effect. For example, it is not always the case that inadequate funding causes worse
conditions. It could be due to the poor quality of care that funding was reduced for a particular
organisation.
164
Different types of relationships between cause and effect
• Direct cause-and-effect relationship: for example, if a university has a set number of students it can
take each year and increases its intake of part-time students, it must reduce its intake of full-time
students.
• Reverse cause-and-effect relationship: for example, poor examination results could be due to poor
attendance, but equally, poor attendance could be due to poor examination results.
• Coincidence: for example, there may be a relationship between the quality of healthcare in a local
authority and examination results in that same area, but it is difficult to know whether one causes the
other.
• Confounding effect: for example, the relationship between quality of health care and exam results
could be due to effective use of resources within the local authority, which may not have been
considered part of the fieldwork.
Source: Adapted from GUID 3920, Box 7
How do you assess your evidence?

There are several techniques you can use to assess your evidence. The nature of your audit and
the information collected will help you determine the most appropriate way to do so. Your audit
team must work systematically and carefully in interpreting the evidence and the data collected.
As stated in Chapter 5, assessing and ensuring the sufficiency and appropriateness of evidence is
also critical throughout the audit. It is an important first step before you assess your evidence to
help determine findings. In addition, combining data from a range of sources, methods and
analysis (corroborating data) allows you to overcome any bias that can come from using a single
source of information. This section describes some common methods for assessing evidence.
Grouping and labelling evidence
One technique to assess evidence is to group and label information to identify logical categories.
To group information, you would place information into logically related groups so that the
information in each group all closely relates to each other. Grouping helps you identify ways in
which information collected from different sources may be connected.
After analysing the relationship between the information in a group, you can then label each
group with a heading: either a phrase or a sentence that expresses this relationship as the main
theme. A label can simply be a heading that expresses what the individual information adds up
to.
165
Your audit documentation can be used to help you populate this information. For example, for
information collected to answer an audit question related to the sufficiency of training for
customs inspectors, you could potentially group the information collected into categories such
as ‘resources’, ‘benefits’ or ‘challenges’. To label the information, you could review the
information contained in that group to say: “Attendance is low at the inspection training
academy.”
Using visual displays or linkages

Another technique is to use visual displays to make connections within and across audit
questions. Options include a mind map, a fishbone chart, or an organisational chart. See Figures
35, 36 and 37, which have a portion of the information completed to give you an understanding
of how you might go about populating the boxes based on your evidence. These options can also
be used during other audit phases, such as the design phase.
Figure 35: Mind map
Use of
canines
Training
curriculum
Customs
Agricultural laws
inspections
Drug
inspections
Source: Adapted from US GAO
A mind map helps visualise and display all the information related to a specific topic or question.
For example, for an audit question related to the sufficiency of training for customs inspectors,
you could use the topic of training curriculum as the central anchor or idea and use each
surrounding box to display one of the topics the curriculum covers.
166
Figure 36: Fishbone chart
Not enough Not enough hands-

instructors on training
Training does
not meet
curriculum
guidance
Outdated training
scenarios
A fishbone chart can be used to graphically identify and organise possible causes of a problem so
that you can develop recommendations aimed at the root cause. Taking the previous example of
the sufficiency of training for customs inspectors, you could use the problem statement that
customs inspection training given to new inspectors does not meet the training curriculum
guidance. You can then use the bones of the chart to describe potential causes, such as challenges
associated with personnel, equipment or policies.
167
Figure 37: Organisational chart
Training does not meet

curriculum guidance
Not enough Not enough

instructors hands-on training
Instructor Not enough Insufficient

attrition is money to equipment
high hire more for hands-on
instructors training
An organisational chart can help you display how each piece of evidence is related to the others.
Using the previous example, you could use the problem statement that customs inspection
training given to new inspectors does not meet the training curriculum guidance at the top of the
organisational chart. You could use the next level of boxes to describe the different instances of
how the training does not meet the guidance.
168
Writing on walls Tips for preparing for a
writing on walls session:
Another way to assess the evidence as a team is a technique sometimes
referred to as ‘writing on walls’. This is a technique where the entire team • Allow at least two weeks
between data collection
and its internal stakeholders and supervisor assemble in a room (or gather and holding a writing on
virtually). With the help of a trained facilitator, the team talks through their walls session so that all
documentation is collected
audit questions and what evidence they have collected that addresses and reviewed prior to the
each of those questions. Teams then visually display the evidence, using session.
• Review all of your audit
sticky notes on a wall or via a computer, so everyone on the team can see
documentation to be
the weight of evidence and what themes develop from that evidence. Over familiar with materials and
pay particular attention to
a few days, the team then discusses the various evidence, often moving those you believe may be
around the notes and developing a visual display of the audit findings. The particularly relevant to your
audit questions.
facilitator plays an important role in asking the team and stakeholders
• Try to keep an open mind: it
about the supporting details of the evidence, the reasons (causes) for any is best not to come to the
deficiencies and the effects. writing on walls session with
preconceived notions of the
findings and
recommendations.
How do you develop conclusions and recommendations? Source: IDI/PAS Development Team
Assessing your evidence may lead to audit findings and, based on these findings, you may be able
to reach conclusions and recommendations. Findings and conclusions must be supported by
sufficient and appropriate evidence.
How do you develop conclusions?
The Standard
The auditor shall obtain sufficient and appropriate audit evidence in order to establish audit findings,
reach conclusions in response to the audit objective(s) and audit questions and issue recommendations
when relevant and allowed by the SAI´s mandate.
Conclusions allow you to make a concise and persuasive argument that action is needed to
address a deficiency or take advantage of an opportunity for improvement and set up the basis
for any recommendations. Conclusions also allow you to: present your opinion anchored in your
evidence; clarify and add meaning to the specific findings, and go beyond restating the findings
that will be presented in your audit report. The conclusions also reflect the auditor’s explanations
and opinion based on these findings; for instance, conclusions might include identifying a general
topic or a certain pattern in the findings or an underlying problem that explains the findings
169
(adapted from GUID 3920/93). When drafting conclusions, it is vital that the audit team critically
consider the conclusions in relation to the audit findings, evidence, and audit criteria. It is also
important to link the conclusions with the audit objective.
Communication is essential for developing your findings because it is important for the auditor
to consider the context, all relevant arguments, and different perspectives before conclusions
can be drawn. For this reason, the auditor needs to maintain effective and proper communication
with the relevant stakeholders within your SAI and the audited entities (adapted from GUID
3920/100-124). This communication is discussed later in this chapter and in Chapter 7.
Conclusions
Check that the conclusion:
✓ states the degree of economy, efficiency ✓ reflects changes over time (for example,
and/or effectiveness through an overall view states whether risk to performance is due to
on aspects of the 3Es or by providing specific increase soon due to new developments);
information on a range of points related to ✓ is balanced in tone, is deduced from the
the 3Es; audit findings and reflects fairly the audit
✓ is clear and concise – you do not need to findings;
repeat all of the findings in the conclusions ✓ provides a clear linkage to the
section; recommendations of the report. Some SAIs
✓ reflects the audit criteria; may not require all conclusions to be directly
linked to a recommendation.
✓ is quantified where possible (for example,
states how far performance has fallen short
of the expected or ideal standard);
Tips for developing effective conclusions
✓ Link the conclusions back to the audit ✓ Make sure that the conclusions flow logically
objective and explain why the audit is from the audit findings.
important. ✓ Do not merely summarise or restate the
✓ Ensure that the conclusions are balanced, findings, but explain their significance and
highlighting the significance (positive and why recommendations are needed.
negative) of the audit findings and the
audited entity’s progress (if any) in dealing
with problems raised.
170
How do you develop recommendations?
The Standard
The auditor shall provide constructive recommendations that are likely to contribute significantly to
addressing the weaknesses or problems identified by the audit, whenever relevant and allowed by the
SAI’s mandate.
ISSAI 3000 addresses recommendations in the reporting stage of an audit. Still, we have included
developing recommendations in this section to help auditors understand the connection
between findings, conclusions, and recommendations.
Recommendations to correct deficiencies and other findings identified during the audit are
developed if needed. It is helpful to show the linkage between your audit findings and
recommendations by using consistent keywords and phrases. The features of a good
recommendation can be represented by the acronym SMART: Specific, Measurable, Attributable,
Relevant and Time-bound. In some circumstances, discussions with the audited entity can help
the team determine the ‘Time-bound’ piece of SMART or timeframes for implementation of a
recommendation.
Any recommendations developed should address causes of the deficiencies identified and help
to improve the audited entities’ programmes, operations and performance, without encroaching
on the audited entities’ management responsibilities. You should also discuss your potential
recommendations with the audited entities before drafting the report, as discussed further in
this chapter and in Chapter 7.
Recommendations are often aimed at eliminating the deviation between the evidence and the
audit criteria. Recommendations are most effective when they clearly demonstrate that they are
worthy of action, reasonable and cost-effective. Such constructive recommendations are
(adapted from GUID 3920/127):
• directed at resolving the causes of weaknesses or problems identified;

• practical;
• value-added;
• well-founded and flow logically from the findings and conclusions;
• phrased to avoid truisms or simply inverting the audit criteria or conclusions;
171
• neither too general nor too detailed. Recommendations that are too general will typically risk
not adding value, while recommendations that are too detailed could restrict the necessary
flexibility of the audited entities. Additionally, SAI policy and procedures may require that
recommendations made to an audited agency may not be so prescriptive and detailed that
the SAI might be seen as consultants as opposed to independent and impartial auditors; and
• addressed to those responsible for taking the actions, and clearly state the actions
recommended.
When possible, consider:

• if any of the recommendations could take priority (be implemented first) over others;
• what resources might be needed to carry out the recommendations;
• if the benefit to be derived from the recommendation is worth the cost to implement;
• how to follow up the recommendations. See Chapter 8 for more information on follow-up.
Recommendation
✓ Think about potential recommendations ✓ Discuss recommendations with the audited

early in the audit process and frequently entity to identify the necessary changes and
ask actors what can be done to improve practical ways of implementing them. This will
performance. lead to a realistic implementation of the
✓ Write the recommendations in a way that recommendations.
allows the Supreme Audit Institution to
evaluate whether they have been
implemented.
Audit findings matrix
One tool you can use for assessing your evidence and developing conclusions and
recommendations is an audit findings matrix, as shown in Figure 38. This tool allows you to
determine whether your findings and recommendations, if applicable, are based on sufficient
and appropriate evidence. A well-developed audit findings matrix can also help as you write your
report.
172
Figure 38: Audit findings matrix template
Audit objective: Clearly and objectively express what the audit is about.
Audit question (the same stated in the audit design matrix): For each audit question (or sub-question), repeat each of
the items mentioned in the table.
Most relevant occurrences identified in the fieldwork.

Situation
found
(Condition)
Criteria Information used to determine if the expected performance of the audited object is
satisfactory, exceeds expectation or is unsatisfactory.
Evidence Result of applying data analysis methods or assessing your evidence. The techniques used to
Finding and analysis handle the information collected during fieldwork and the results achieved can be indicated.
Reasons for the situation found.

May be related to operation or design of the audit object.
Causes May be out of the control of the manager.
Any recommendations should be related to the causes.
Consequences related to causes and corresponding evidence.

Effects
It may be a measure of the finding’s relevance.
Is the evidence Consider the evidence you have for each element of the finding and whether it is sufficient
sufficient and and appropriate.
appropriate? If not,
what remaining work If your current evidence is not sufficient and appropriate for each element, what remaining
is necessary to work is necessary to address any gaps in the evidence?
address any gaps?
Good practices Actions identified that lead to good performance.

May support the recommendations.
Recommendations Proposals to address the causes (or deficiencies) identified.

Source: Adapted from US GAO and SAI Brazil
173
Figure 39 shows an illustration of one finding of an audit findings matrix for a performance
audit.
Figure 39: Illustration of one finding of an audit findings matrix

Audit objective: Examine growing concerns about sexual violence – unwanted sexual acts – in the United States,
particularly involving populations such as university students, incarcerated individuals and military personnel.
Audit question: To what extent are government agencies addressing any challenges posed by the differences across
existing data collection efforts on sexual violence?
Situation Agencies’ efforts to lessen differences between data collection on sexual violence have
found been fragmented and limited in scope.
(Condition)
Criteria The Committee on National Statistic’s Principles and Practices for a Federal Statistical Agency
requires federal agencies that produce similar federal statistics with different missions to:
(1) coordinate and collaborate to meet current information needs; and
(2) provide new or more useful data than a single system can provide.
Finding
Evidence
and analysis There are five agency efforts that are intended to increase harmonisation across data
collection efforts.
Coordination for these efforts is bilateral (generally involve two of the ten data collection
efforts at a time), and scope is limited.
Office of Management and Budget does not plan to form an interagency group on
Causes harmonising data on sexual violence. They cited that they plan to focus on other priorities
instead, such as redesigning the National Crime Victimization Survey.
Sexual violence data is inconsistent, incompatible and there is confusion about the data.
Effects There is a lack of understanding about the scope of sexual violence in the United States.
Is the evidence Yes.

sufficient and
appropriate? If not,
what remaining work is
necessary to address
any gaps?
Good practices None.
Recommendations To the Director of the Office of Management and Budget to establish an interagency group on
sexual violence statistics that considers the differences across the data collection efforts to
assess which differences enhance or hinder the overall understanding of sexual violence in the
United States.
Source: Adapted from a US GAO audit
174
How do you prepare for drafting your report?
Discussions within your SAI
After you have developed your findings, conclusions and recommendations, as applicable, it
is helpful to describe these findings, refine the key messages and themes you want to
emphasise, and reach an agreement within your SAI to prepare for drafting your report. You
may want to consider holding a meeting with all the auditors, internal stakeholders and
managers that have worked on the performance audit so that agreement is reached about
the audit findings. Another option is to discuss emerging findings as part of your ongoing work
and interaction with the members of the audit team, internal stakeholders and managers.
To reach an agreement about your audit findings and prepare for report drafting, the audit
team can discuss the findings for all audit questions, considering the strength and reliability
of evidence for each answer, and identify and address any ambiguities or uncertainties within
the evidence. For any uncertainties, it may be necessary to collect additional evidence.
Discussions with the audited entities
As a final step before you begin to draft your report, it is important that you communicate
and discuss your audit findings with the audited entities. This may help you determine if any
refinements may be necessary based on the audited entities´ perspectives and any actions
that have occurred since you collected your evidence. If you have been in close
communication with the audited entities during the study, this step will likely be smooth, as
there would probably not be any surprises.
You can use the audited entities’ initial reaction to:
• gauge if your conclusions are reasonable;

• request additional evidence, as needed;
• identify and correct any factual errors in the draft audit findings;
• add new material to the draft audit findings to reflect the audited entities’ view; and
• refine any recommendations (if they could be more specific, feasible and thereby more
likely to be implemented by the audited entity).
Keeping your independence and professional scepticism in mind, you may need to make
changes to your prospective draft report following these discussions with the audited entities
and the receipt of any additional evidence. This is not a bad thing – it is all part of the process
of producing a high-quality report. It is essential that all such changes are based on good-
quality evidence.
175
When developing audit findings, conclusions and recommendations
(if applicable), remember to:
… reconsider the initial assessment of risk in … analyse and confirm causes – why there is
light of the evidence collected and a deviation from the criteria – if a mismatch
determine if additional audit work needs to be between the criteria and the evidence was
performed; identified;
… work systematically and carefully to analyse … identify either positive or negative effects if
your evidence and the data collected, a mismatch between the criteria and the
ensuring that the audit findings are put in evidence was identified;
perspective and respond to the audit … ensure that any conclusions and
objective(s) and audit questions;
recommendations you develop (if
… ensure that audit findings are objective, fair applicable) flow logically from the audit
and balanced – maintain independence, findings and are balanced and reasonable;
include both good and bad points and give and
credit to the audited entity when it is due;
… communicate and discuss your preliminary
… consider the materiality of the findings and findings, and your conclusions and
apply professional judgement in interpreting recommendations (if applicable), with the
how the findings affect the audited entity’s audited entity(ies).
performance;
176
Chapter 7
How do you write a performance audit report?

Writing the report
• Establish a report structure that will effectively • After receiving SAI management approval,
communicate the audit results. finalise and publish the report.
• Draft the report in accordance with your SAI • Communicate the audit results to the relevant
guidance. parties.
• Obtain the audit entity’s comments on the
draft report.
This chapter explains how you prepare and draft a performance audit report. The purpose of
a performance audit report is to communicate the results of the audit to the legislative
authority, the audited entities and the wider audience. Whether you are publishing in print
or only in electronic form, the same high-level principles will apply.

• How do you develop a performance audit report?
• What are the main attributes of a performance audit report?
• How do you create a logical report structure?
• How do you ensure the quality of the report?
• How do you consider audited entities or third party comments?
• How do you publish the final report and communicate the results?
How do you develop a performance audit report?

A good audit report clearly and objectively lays out the material findings and conclusions of
the audit questions, and if appropriate, provides practical recommendations. The report
should be easily intelligible to the intended reader, who should clearly understand what was
done to perform the audit, why it was done, and how. They should also be able to understand
the context of the audit topic.
177
SAIs take different approaches to allocating the task of drafting the report itself. Some SAIs
may divide the work between members of the audit team, while others have team members
who specialise in drafting. The person who drafts the report will not always be the person
who collected the audit evidence. If you have prepared a clear report structure that shows
where each audit finding fits, the process will be smoother and less prone to error.
If more than one person prepares the draft, you need to allow sufficient editing to ensure the
entire report is consistent in style and tone. It is important that your supervisor review the
draft, looking in particular for areas where the evidence or logic appears weak. You might also
consider a review from someone outside of the team to ensure the evidence and logic clearly
support the conclusions. Reviews from outside the team can also help ensure clarity and
independence.
What are the main attributes of a performance audit report?
The Standard
Auditors should strive to provide audit reports that are comprehensive, convincing, timely, reader-
friendly and balanced.
As you write the report, you need to keep in mind the five main attributes of performance
audit reports. These attributes have to be present in a performance audit report regardless
of the structure chosen. You can find further guidance in GUID 3920/108-124.
Comprehensive
It is important to be comprehensive in that you include all the information and arguments
needed to address the audit objective and audit questions in the report. At the same time,
the report has to be sufficiently detailed to understand the subject matter and the audit
findings and conclusions (ISSAI 3000/117). Most importantly, you should make sure the report
has sufficient and appropriate evidence to support the findings, conclusions and
recommendations (if applicable) about the audit objective(s) (GUID 3920/114). These
elements are discussed in more detail later in this chapter.
Convincing
The reader has to be convinced by your argument in the report that leads to the conclusions
and recommendations (if applicable). To be convincing, a performance audit report needs to
178
be logically structured and present a clear relationship between the audit objective(s) and
audit questions, audit criteria, audit findings, conclusions and recommendations. Chapter 6
provides more details on developing findings based on sufficient and appropriate evidence.
The report also needs to present the audit findings persuasively, address all arguments
relevant to the discussion and be accurate. Accuracy requires that the audit evidence
presented and the audit findings and conclusions are presented in a neutral, fact-based
manner. (ISSAI 3000/118)
The SAI has two main goals when aiming to produce a convincing performance audit report:
• Logic. You have to map the logic of the argument that leads to the conclusion and
recommendations (if applicable). There should be a clear linkage from the audit criteria via
findings to the conclusions and recommendations.
• Accuracy. Inaccurate reports can damage the credibility of the SAI. One inaccuracy in a
report can cast doubt on the validity of the entire report (or even the SAI) and can divert
attention from the substance of the report.
Timely
A performance audit report needs to be issued in a timely manner to make the information
available for use by management, government, the legislature and other interested parties
(ISSAI 3000/119). In some cases, the SAI may also choose to report early on a new programme,
with plans to return to the topic to assess progress.
The audit report is intended to result in improvements within the audited entities. These
improvements are expected to enable the entities to achieve its objectives more efficiently
or effectively.
Reader-friendly
SAIs should report objective, fact-based information simply and clearly, using language
understood by all their stakeholders (INTOSAI-P-12/Principle 4). While specific
communication styles and preferences may vary between different countries and cultures,
aim to always keep the tone of your report professional and neutral.
The report needs to be concise but with sufficient evidence (see the discussion on
comprehensiveness earlier in this chapter). Conciseness will ensure that the volume of the
report is no longer than it needs to be, which will ensure clarity and help to more effectively
convey the message of the report (ISSAI 3000/120). A long report, however well-written, can
179
be intimidating or off-putting to readers. If a lengthy report is unavoidable, you may want to
consider using appendices or publicising a standalone summary of the main points.
To produce a reader-friendly reporti, it helps that you know the audience, understand its
needs and write the report accordingly (GUID 3920/120). As discussed earlier, consider using
readers outside the audit team to check if the report is easy to understand. You can also use
simple automated readability analysers to get a basic assessment of the complexity of the
text. These tools use metrics such as average sentence length and sophistication of the
vocabulary to suggest the reading level needed to understand the report. They can be found
in open source or as part of word-processing programs.
Reader-friendly
✓ Use short rather than long sentences. ✓ Use non-text visual aids (such as pictures,
✓ Use simple sentence construction (the illustrations, charts, graphs, maps).
simplest being subject – verb – object). ✓ Avoid technical jargon and complex,
✓ Use active voice. seldom-used words.
✓ Break up the text with the use of headings. ✓ Avoid excessive use of cross-referencing and
acronyms.
Another way to be reader-friendly is to include graphics or visuals throughout the report.

Including visual displays of information can help the reader quickly understand complex
concepts or see how ideas relate to each other. They can also eliminate the need to write out
extensive descriptions of processes or events. Photographs may also be useful to include if
they are pertinent to the subject matter. A busy reader will find a report filled with visual
displays of information easier to read. Figure 40 provides examples of visuals in performance
audit reports.
180
Figure 40: Examples of visuals displays of information in performance audit reports
Commercial airport categories for United States (US) airports based on boardings of US
passengers (2017)
Commercial Minimum required Annual passenger boardings Number of airports

airport percentage of total per airport category
category annual passenger
boardings
Large
hub 1% or more 72.0% 617,598,283 30
Medium At least 0.25%, but 16.2% 31

hub 138,949,064
less than 1%
Small At least 0.05%, but

hub 8.3% 71,157,137 70
less than 0.25%
More than 10,000,

Nonhub but less than 0.05% 3.4% 28,881,284 255
Commercial At least 2,500 and no

service 0.1% 627,545 125
more than 10,000
non-primary
Source: GAO analysis of the Federal Aviation Administration’s 2017 enplanement data, AIRPORT INFRASTRUCTURE:
Information on Funding and Financing for Planned Projects, 2020, GAO-20-298
Note: The term ‘hub’ is defined in federal law to identify commercial service airports as measured by passenger boardings, and the airports
are grouped into four hub categories. (49 US Code Sections 40102 (29), (31), (34) and (42).)
Reported incidents of child abuse (physical, sexual or emotional abuse or neglect), by

Department of Defence (DOD) criteria for abuse, fiscal years 2014–2018
Army
Navy
Marine
Corps
Air
Force
0 5,000 10,000 15,000 20,000
Reported incidents
Met DOD’s abuse criteria Did not meet DOD’s abuse criteria
Source: GAO analysis of Family Advocacy Program data, CHILD WELFARE: Increased Guidance and Collaboration
Needed to Improve DOD's Tracking and Response to Child Abuse, 2020, GAO-20-110
Inclusiveness is also an important component of reader-friendliness. Readers will tend to give

more credibility to your report if they see that you have addressed their particular concerns.
For example, if you are dealing with a programme that has differing impacts in different parts
of your country, you can draw readers in by using maps and other graphics to show this
varying impact across different regions or sectors of society.
181
Balanced
Being balanced means that the performance audit report is impartial in content and tone. You
should present all audit evidence in an unbiased manner and be sure to report both positive
and negative findings. Often, auditors tend to focus on problems, which lead to
recommendations for improvement. Explaining the impact of such problems does help the
reader understand the significance of the problems, which encourages corrective action
(GUID 3920/123). While this process is important, it is equally important to provide the full
picture of the audit topic or activity. If the audited entity is doing something well, be sure to
report that and areas in need of improvement. Be aware of the risk of exaggeration and
overemphasis on deficient performance by the audited entities. (ISSAI 3000/121)
Balance
✓ Present findings objectively and fairly, in ✓ Be complete. Include points both positive
neutral terms, avoiding biased information or and needing improvement. Give credit
language that can generate defensiveness where it is due. Including positive aspects
and opposition from the audited entity. may lead to improved performance by
✓ Present different perspectives and viewpoints other government organisations that use
on the topic. the report.
✓ Facts must not be suppressed, and minor
shortcomings should not be exaggerated.
How do you create a logical report structure?

Considering the five main attributes, you will need a logical report structure to communicate
your audit findings, conclusions and recommendations to readers. In some cases, each of your
audit questions will map neatly to a separate chapter in the audit report. However, this does
not happen in all instances. For example, if the findings from two separate audit questions
are interrelated, it may be more appropriate to present them in the same chapter. In other
instances, you may choose a logical order structure based on the materiality of different
findings or the chronological order of events.
Regardless of the structure chosen, it is always good practice before drafting to map out an
outline for the report. This approach will strengthen the logical flow of the report and reduce
the risk for needless duplication of content. An effective structure will grab the reader’s
attention, communicate complex issues clearly, and provide a clear interpretation of the
results (GUID 3920/121). The report will also need enough context or background information
for the reader to understand the subject. For example, the reader may need to understand
the roles and responsibilities of those involved in the audit topic, the amount of money spent
and the government’s goals for the audited area.
182
Report formats will take many different shapes and forms, depending on the SAI and the audit
work. You need to keep the audience in mind as you develop an appropriate report structure.
Some SAIs find it helpful to use a ‘Dinner Party’ approach to build a reader-friendly report
structure. The Dinner Party approach imagines that you are speaking to fellow guests and
only have a short time to hold their attention. What are the main things they need to know
about what you have found during your audit? Once you have established these interesting
conclusions, you can build up the finer detail that supports these conclusions. (GUID
3920/121)
The information below discusses common segments of a report.
Title
A good title communicates the topic (or the message) of the report. In some message-
oriented titles, the title may preview the recommendations:
• An example of a descriptive title: U.S. Efforts to Combat Trade-Based Money

Laundering (GAO-20-314R).
• An example of a message-oriented title: Weaknesses in Cybersecurity Management
and Oversight Need to Be Addressed (GAO-20-199).
You may also want to decide whether or not to mention the name of the audited entities in
the title. For example, if your report covers the work of several different ministries, you might
omit their names from the report title for the sake of brevity.
Audit report title

Without the name of the responsible entity
Protecting Canadian Critical Infrastructure Against Cyber Threats (2012 report of the Auditor
General of Canada)
With the name of the responsible entity

Inland Revenue Department: Managing child support debt (2010 report of the Auditor General of
New Zealand)
Table of contents
A good table of contents displays the report's structure, allowing readers to easily find the
sections in which they are interested. Figure 41 shows one such approach.
183
Figure 41: Contents page using question-based headings
Heading Page
Summary and recommendations 7
Background 7
Conclusion 8
Supporting findings 8
Recommendations 10
Summary of entity response 10
Key messages from this audit for all Australian Government entities Audit findings 11
Audit findings 13
1.Background
The Australian Taxation Office’s (ATO) compliance, dispute and debt collection activities for
14
small business
14
Rationale for undertaking the audit
19
Audit approach
2.Organisational arrangements for managing small business tax debt

Does the ATO have strategies and processes to support the effective management of small
19
business debt?
Does the ATO have effective processes for coordination across small business compliance,
21
dispute and debt activities?
Does the ATO use international practice, stakeholder views and continuous improvement to
21
inform changes to debt strategies and processes for small business?
3.Consistent management of small business debt

Does the ATO have processes that support consistency in progressing debt towards firmer, and
28
stronger action?
36
Does the ATO have effective controls to ensure consistent firmer and stronger debt action for
small business tax debt?
4.Performance monitoring and reporting

Is there effective monitoring and reporting of small business tax debt, including tax debt arising
39
from compliance activities?
Does the ATO have a sound performance framework in relation to the effective, efficient and
44
consistent recovery of small business tax debt, including debt arising from compliance
activities?
Appendices
54
Appendix 1 Australian Taxation Office response
54
Appendix 2 ATO firmer and stronger actions
58
Appendix 3 Issues raised by the ABC Four Corners investigation
69
Appendix 4 ATO Quality measures and indicator
70
Source: Management of Small Business Tax Debt, Australian National Audit Office, 2019
In Figure 41, the SAI used question-style headings for the parts of the report that cover audit
findings. Some SAIs prefer to use the contents page as a summary of findings. To do so, they
use headings that are one-sentence summaries of their audit findings. Figure 42 provides an
example of this approach.
184
Figure 42: Contents page using descriptive headings
Heading Page
Summary 5
Recommendations 11
Part 1
The Welsh Government has a plan for waste prevention but has focussed more attention and 13
resources on recycling
The Welsh Government has focussed successfully on increasing municipal recycling, but waste 14
prevention has generally had a lower profile despite some important initiatives
While the Waste Prevention Programme reflects common practice, there are opportunities to learn 17
from approaches elsewhere and to make further use of legislation and financial incentives
The Welsh Government has provided councils with significant funding for their municipal waste 23
management services, but this has mostly supported recycling, with very little of it spent on waste
prevention
Between October 2015 and September 2018, the Welsh Government gave £13 million to three not- 26
for-profit organisations with objectives that include, but are not necessarily limited to, waste
prevention
Part 2
The data used by the Welsh Government to measure performance against its ambitious waste 30
prevention targets is of variable quality and indicates mixed progress
The Welsh Government has good data on municipal waste, but the data it has on most other wastes 31
is limited
The data that is available on the amount of waste produced indicates mixed progress to deliver the 35
Welsh Government’s ambitious waste prevention targets
Appendices
Appendix 1 – Audit methods 43
Appendix 2 – Key elements of the Welsh Government’s Waste Prevention Programme and the waste 44
sector plans 45
Source: Waste Management in Wales – Preventing Waste, Wales Audit Office, 2019
Executive summary
The executive summary is a short chapter designed to provide a quick synopsis of the main
points and key messages of the report. Many people only read this section of the report; thus,
it is important that it is written clearly and that it concisely summarises the most important
parts of the report. Typically, an executive summary includes an explanation on why the audit
was carried out, brief information on the subject of the audit and the audited entities, the
audit objective and questions, the scope, the methodology, the key findings, the conclusions
and the recommendations.
Not all SAIs use an executive summary; some summarise the audit report in other ways. For
example, the US GAO uses a one-page abstract instead (see Appendix 18). Where used, a
good summary will:
185
• reflect accurately what is in the rest of the report;
• be concise without omitting important audit findings;
• guide the reader as to the significance of the audit questions and their answers;
• show the reader which parts of the main report support each key audit finding;
• work as a standalone document;
• only include material covered in the main report; and
• mention important contextual information such as previous audits or the legislature´s
coverage of the topic.
When appropriate, the executive summary may include a graphic or visual to help convey the
message.
An effective executive summary answers the fundamental questions the reader will have
about the audited topic and the SAI’s assessment of performance. For example, the United
Kingdom’s National Audit Office guidance on drafting audit summaries asks auditors to
answer the following list of questions, as appropriate:
• Assessing performance. What would good value look like in the context of this study?
What comparator or counterfactual has been used?
• Where the comparator is operational good practice. How has good practice been
determined?
• Quantification. Are the total resources at stake and achievements quantified? Are costs
and benefits presented in a way that supports the conclusion on performance?
• Causality. Is it possible to attribute value or benefits in the system to the specific spending
being examined? In interpreting data, what other factors may have affected outputs and
outcomes?
• Uncertainty. What are the risks and uncertainties relating to data used in the report? All
data is subject to uncertainty, and it is reasonable to state explicitly the level of
uncertainty.
The length of the executive summary is usually proportionate to the length and complexity
of the main report. However, a typical summary tends to be less than three pages long.
Appendix 19 illustrates an executive summary.
186
Executive summary
✓ Build an executive summary from summary ✓ It is sometimes useful to think of: the executive
paragraphs and sentences within the main summary as being written for the Public
report – this will ensure that the summary is Accounts Committee or oversight legislative
consistent with the report. committee; the report as being written for the
audited entity; and the appendices as being
written for those academics or specialist staff
with an interest in the field and the detailed
methodology.
Introduction
The introduction of the report provides the context of the performance audit, helping the
reader to understand the audit. Typically, SAIs use the introduction section of the report to
describe the audit topic but not to provide audit findings.
The introduction does not need to be overly long and detailed. For example, Figure 43 shows
how you might use a simple graphic in the report’s introduction to quickly explain the
responsibilities of various audited entities. If the reader needs more detail, you can provide it
in an appendix or in a separate background section; or you can indicate where the reader can
obtain further information.
Introduction
✓ Include sufficient context for the reader to ✓ Tell the reader why you are reporting on this
know how the audited activity works and is activity now.
managed, but not so much detail that they ✓ Use appendices, cross-references or a
are tempted to skip the section. bibliography section to point reader towards
✓ Consider using a simple diagram to illustrate further details.
who is responsible for which audited activity.
187
Figure 43: Using a simple graphic to illustrate the responsibilities of audited organisations
The responsibilities of the Border Delivery Group and departments
Inter-ministerial group on borders Cabinet Secretary and Chief Executive of the

Civil Service
Cross-government ministerial group, which provides Support the Prime Minister and ensure the effective
scrutiny and oversight. running of government.
HM Treasury
Ensures that appropriate funding for EU exit is in place.
Border Planning Group (BPG) and Border Planning Executive Group (BPEG)
Oversee and assure plans for managing the impact of EU exit at the border.
It is co-chaired by HM Revenue & Customs’ Chief Executive and Home Office Second Permanent Secretary.
Border Delivery Group

Department for Exiting Border Delivery Group (BDG)
Steering Groups –
the European Union stakeholder
(DExEU) Responsible for scoping, planning,
coordinating and ensuring delivery of the engagement
necessary change plans to ensure the Support BDG and BPEG in
Provides data from their strategic oversight
border works effectively after EU exit.
departments based on and assurance of plans to
the monthly returns. ensure coordinated
Team is led by Director General Border
Delivery and works across departments. communication with
stakeholders.
Government departments
Responsible or accountable for delivery at the border. Key departments with these responsibilities are: HM Revenue
& Customs; Home Office including Border Force; Department for Environment, Food & Rural Affairs; and Department
for Transport.
Source: National Audit Office UK – analysis of departments’ documents
Audit objective(s) and questions
It is important that your performance audit report has to describe the audit objective(s) and
the audit questions. Readers need this information to understand the purpose of the audit,
the nature and extent of the audit work performed, and any significant limitations on the
audit objectives, scope and methodology. See Chapter 4 for examples of audit objectives and
questions.
Audit scope and methodology
Different readers have different needs and expectations from the audit. The audit scope helps
the reader understand:
188
• what to expect from the report;
• what use can be made of the findings and conclusions; and
• the degree of reliance they can place on the findings and conclusions.
Be sure to tell the reader about what is in and out of scope in your audit approach, what time
period or geographical area is covered, and who is the subject of the audit. It is important to
tell the reader if the report focuses on a narrower set of audited activities than might be
implied by the report title. For example, if a report entitled Protecting Wetlands does not
cover activities in all national wetlands, you will need to explain the particular focus of the
report.
Also, describe the audit methodology used to address the audit objective(s).
An excerpt of a description of a performance audit report methodology
“To examine the characteristics of Federal Aviation Administration-certificated mechanics and

repairmen, we analyzed cumulative FAA data as of December 2018 for demographic
characteristics such as age and sex. To examine the employment characteristics of aviation
maintenance workers – such as wages and unemployment – we analyzed Bureau of Labor Statistics
Current Population Survey data for selected labor market indicators from 2013 through 2018, and
we reviewed all 50 states’ most recent Workforce Innovation and Opportunity Act plans.”
Source: US GAO report AVIATION MAINTENANCE: Additional Coordination and Data Could Advance FAA Efforts to Promote a Robust, Diverse Workforce, 2020, GAO-20-206
You may describe the methodology briefly in the report body, with more details in an
appendix. See Appendix 20 for a description of the mentioned report’s scope and
methodology.
The main body of the report has to mention at a minimum and in a concise form:
• the audit methodology and approach;
• the sources of data;
• the data gathering and analysis methods used; and
• any limitation on the data use.
It is often important to provide more details to the reader about the methodology or any data
used in the report. Thus, providing more details, often in an appendix, could be appropriate.
For example, you may provide information on:
• what you have done to establish the reliability of the data;
• if there are methodological limitations that the reader should know about, for example,
limitations in the data and analysis and how they should be interpreted;
• if there were limits on the data and other evidence you could access; and
• if any trends you identify in your quantitative analysis are statistically significant.
189
Audit criteria and sources
It is important to state what the audit criteria are, how they were developed and what the
sources were. By drawing attention to the audit criteria, you clarify the standard against which
you are judging performance. If you are clear about your sources, audit criteria, methods and
assumptions, you will help the reader to judge how much weight to give the evidence and
conclusions in your report. (GUID 3920/115)
Audit criteria are not always readily available in performance auditing. In such cases, the audit
team needs to develop the criteria and discuss them with the audited entities. If the audited
entities does not agree with the criteria, the SAI has the final responsibility to set them. In
case of significant disagreement on criteria developed by the SAI, the auditors need to
consider the risk that the audited entities will dispute findings where the auditors only can
refer to themselves as the source of criteria. This topic is discussed in more detail in Chapter
4.
Audit findings
The Standard
The auditor shall ensure that the audit findings clearly conclude against the audit objective(s) and/or
questions, or explain why this was not possible.
As discussed in detail in Chapter 6, it is important when you draft the report that the reader
understands how the audit findings relate to the audit criteria and the evidence gathered
during fieldwork. Many reports make this linkage clear by organising the findings according
to the audit questions. Each audit question becomes a section or a chapter of the audit report,
which contains the relevant findings. If different findings are interlinked or not, it can be
appropriate to develop conclusions in the same chapter or as a separate chapter. Regardless
of the organisation method chosen, it is important that the reader clearly understands the
condition, criteria, cause and effect for any findings, as discussed in Chapter 6.
190
Example of a description of a finding in a performance audit report
• No data governance structure to manage all
We identified deficiencies associated with the
drug transaction data. Although DEA has
Drug Enforcement Administration’s drug
guidance, policies and procedures for the use
diversion efforts, including the following:
of some information systems, it has not
established a formal data governance
• Limited proactive and robust analysis of
structure to manage all data it collects and
industry-reported data. While DEA’s current
maintains, which are integral to its diversion
data systems are not designed to conduct
control activities. A data governance structure
real-time analysis, and it conducts some
is defined as an institutionalised set of policies
analyses of industry-reported data, such as in
and procedures for providing data
response to requests from its field division
governance throughout the life cycle of
offices, DEA could conduct more analyses
developing and implementing data
using automated computer algorithms to help
standards. Industry and technology councils,
identify questionable patterns in the data. For
domestic and international standards-setting
example, DEA could analyse data to identify
organizations, and federal entities endorse the
unusual volumes of deleted transactions or
use of a governance structure to oversee the
unusual volumes of drugs disposed of rather
development, management, and
than sold. It could also analyse data to identify
implementation of data standards, digital
trends in distribution or drug purchases in a
content, and other data assets. While DEA
given geographic area. Other analysis DEA
began efforts to develop a governance
could perform is to look for unusual patterns
structure, it is in the early stages of
when comparing drug orders in one
development and does not have additional
geographic area with other nearby areas.
details or documentation of its efforts. An
These analyses could potentially help DEA
effective data governance structure could
proactively identify suspicious activities or
help DEA ensure its important data assets are
registrants that may warrant investigation.
consistently and fully utilised.
Source: US GAO report DRUG CONTROL: Actions Needed to Ensure Usefulness of Data on Suspicious Opioid Orders, 2020, GAO-20-118
Also, as discussed earlier, using graphics and tables can significantly enhance the readability
of the report.
Conclusions
Many reports include conclusions that summarise the findings and information presented in
the report, as discussed in Chapter 6. There are many ways to write this section, depending
on the SAI’s report style and the audience’s needs. Figure 44 presents illustrations of audit
report conclusions from three different SAIs.
191
Figure 44: Illustrations of audit conclusions
Performance Audit objective. To assess whether EU humanitarian aid for education was
audit report effective in helping children and was delivered efficiently.
from 2021 on Audit conclusion. Overall, EU aid helped children in need and projects achieved
European Union their expected results. However, they did not reach enough girls. In addition, most
humanitarian of the projects in our sample were too short compared to children’s educational
aid for needs, decreasing efficiency. The findings from our examination of 11 projects are
education. summarised in Annex II. 68 projects were relevant and well-coordinated, and the
European Court commission addressed the problems it identified during monitoring visits. Projects
of Auditors, achieved most of their planned results. However, the commission made limited use
2021. of the results of its Enhanced Response Capacity projects. In addition, the
commission did not target enough girls, even though they faced greater
disadvantages. Furthermore, several of the projects did not reach the targeted
proportion of girls.
Performance Audit objective. To assess whether measures implemented by the ministry were
audit report effective to support sustainable artisanal fishery.
Moving towards Audit conclusion. The ministry has taken laudable initiatives in devising and
sustainable maintaining a wide range of interventions targeting artisanal fishermen
artisanal fishery individually, collectively at community and national level through preservation
in Mauritius. SAI and protection of lagoon ecosystems. All these interventions are aligned to SDG
Mauritius, 2018. 14B and the FAO Code of Conduct for Responsible Fisheries, which promote
sustainable artisanal fishery.
Performance Audit objective. To assess whether the Ministry of Agriculture, Livestock and
audit on the Fisheries (MALF), President’s Office Regional Administration and Local
hygiene control Government (PO RALG) and the Ministry of Health, Community Development,
in meat Gender, Elderly and Children (MoHCDGEC) have efficient and effective hygiene
production practices and control mechanism in meat production process to ensure safe and
process. SAI wholesome meat is delivered to the public.
Tanzania, 2016.
Audit conclusion. There is no assurance that the meat delivered to the public in
some of the visited slaughter facilities is safe for human consumption, as meat
inspection and sanitary controls in many slaughter facilities are unsatisfactory. This
is because the hygiene control mechanisms are not effectively and efficiently
managed by (MALF), (PO RALG) and (MoHCDGEC).
Source: ECA, SAI Mauritius and SAI Tanzania
Recommendations
Many reports contain recommendations to the audited entities. You will need to balance the
way that you describe the recommendation; it needs to be clear enough to avoid ambiguity
but not so specific that it encroaches on management’s responsibility. Chapter 6 discusses
the development of recommendations in detail. More guidance on recommendations can be
found in ISSAI 3000/127-128 and GUID 3920/125-128. Figure 45 provides examples of SAI
recommendations.
192
Figure 45: Examples of recommendations in select SAI performance audit reports
Improvements The Department is supporting the introduction of trauma networks by strategic

in trauma health authorities. However, given the lack of progress made in improving major
networks trauma services over the last 20 years, we recommend the following actions:
a. Primary care trusts and ambulance trusts develop and implement triage
protocols to determine which emergency departments seriously injured patients
have to be taken for treatment. This work has to be coordinated by strategic
health authorities.
b. Primary care trusts use their commissioning powers to require all acute and
foundations trusts with emergency departments that receive trauma patients to
submit data to TARN. The purpose is to use the data collected to inform the
ongoing development of trauma networks.
c. Strategic health authorities with hospital trusts develop protocols for the
transfer of patients requiring specialist care or surgical procedures not available
at the receiving hospital.
Coordination of We recommend to the Ministry of Women to:

actions on a. Strengthen its governance systems by revising the legal framework governing
elimination of the elimination of violence against women, improving coordination between
violence NGO’s and relevant government authorities and improving government
against women commitments to international agreements.
b. Coordinate effectively with key relevant stakeholders to strengthen the
application of the legal framework on violence against women.
c. Improve and strengthen its monitoring and reporting arrangements.
Decision For informed decision making while responding to the domestic violence cases,
making for we recommend to the Ministry of Women to ensure:
domestic 1. Access to the background information (if the violator had been convicted, use
violence cases of drugs, possession of firearms etc.) of the domestic violence case for the
responding police officer, to ensure correct risk assessment on the spot and due
protection of the victim.
2. To aggregate defending as well as restraining orders in the same database to
enable swift and effective response to the violations.
3. For early detection of domestic violence, the commission with the responsible
parties to elaborate and implement early detection system for domestic violence
cases within the activities of routine medical checks for children, screening for
women etc.
Government The SAI recommends that the Department for Education:

Assistance to a. crafts appropriate performance indicators to determine the extent the
Students and programme improved access to quality education and decongest the public
Teachers in schools;
Private b. ensure that the programme prioritises the underprivileged;
Education c. establish the GASTPE Composite Team and, thereafter, a dedicated and
(GASTPE) functional office to manage the Program;
d. develop its Information Systems Strategic Plan in order to integrate all the
relevant Information Technology (IT) systems, including those purportedly owned
by Private Education Assistance Committee.
Source: NAO, SAI Fiji, SAO Georgia, and SAI Philippines
193
Abbreviations
For some topics, you may find it difficult to avoid using unfamiliar abbreviations and technical
terms. You can help the reader by providing a glossary of terms and a list of abbreviations at
the beginning of the report or in an appendix.
How do you ensure the quality of the report?
Quality control procedures need to be an integral part of the conduct of each performance
audit to minimise the risk of error and drive consistency in conduct (GUID 3910/102). There
are several ways that SAIs can help ensure that the reports they produce are of high quality.
These include a process to map the evidence of the report back to other sources, obtaining
reviews of the report and obtaining comments from the audited entities before publishing
the final report. ISSAI 140 provides additional guidance on quality control.
How do you map evidence to sources?
It is good practice to produce a data reconciliation or equivalent audit trail that shows the
source of all the numbers, facts and judgements that appear in your published report before
publication. Doing so will reduce the risk of error in the report and make it easier to respond
if the facts are questioned by reviewers, audited entities or third parties. This should be kept
as part of your audit work papers. Figure 46 shows an example of how to prepare a data
reconciliation for your report.
194
Figure 46: Example of a data reconciliation document for a performance audit report
Paragraph Reference Explanation/source
2.3 In June 2019, the Minister for In her address to Parliament on 3 June
Education announced that other 2019, the Minister for Education said,
planned schemes to construct new “It is this Government’s priority to build
colleges would be paused. the best possible colleges for our
students. Some faults have been
identified in the first of the new wave
we have built. We are therefore going
to learn the lessons from these early
problems before we proceed with
building the remaining facilities.”
[Parliamentary Transcripts, June 2019]
Figure 8 provides the breakdown of
2.4 The delay will cost the Ministry $15 this $15 million total. The source is the
million. Ministry’s Project Plan v3.7 of May
2019.
The $15 million total comprises ...
Source: NAO, SAI Fiji, SAO Georgia, and SAI Philippines
What is the report review process?
In Chapter 2, we discussed the need for SAIs to operate quality checks as part of their quality
control and assurance framework. Before publication, SAIs put their revised draft reports
through internal quality controls. SAIs will have their preferences regarding quality review
procedures, but the review is typically conducted by:
• managers at different levels within the SAI;

• communications experts;
• an internal SAI office that is independent of the audit team; and
• possibly, an external expert. For example, you might request a review by an academic
professional, a methodologist or another expert with in-depth knowledge of the activity
you are auditing.
Such reviews provide you with independent assurance that the report is fair and balanced.
Reports that are fair and balanced:
• treat the gathered evidence objectively, avoiding bias and prejudice;
• give due weight to both positive and negative evidence that is relevant to the audit
objective(s) and questions; and
• present the evidence in a way that is not misleading or likely to have the reader draw an
inaccurate inference from it.
It is important that the reviewers have the necessary skills to make an independent
professional judgement. Chapter 2 provides more details on applying professional judgement
and professional scepticism.
195
As discussed earlier, SAIs also commonly pass their final draft through reviews by senior
management, copy editors, and communications specialists. Figure 47 shows how one SAI
organises quality control of its draft reports.
Figure 47: How NAO Tanzania carries out quality control reviews of its audit reports
Quality control
Quality cannot be imposed by reviewers but is The SAI also asks subject matter experts to
something embedded in the whole process of review all draft reports before they are
performance auditing. However, quality control published. The experts are selected among the
review is one important part of this process. In renowned experts in that field with extensive
SAI Tanzania, as in all other SAIs, the quality of theoretical and practical experience on the
performance audit reports is considered to be subject matter under audit. It may, for example,
paramount. It takes a long time to develop trust, be professors from higher learning institutions,
but it can easily be ruined by inadequate retired civil servants or any other expert who
information or poor analysis in single reports. The have got no vested interest with the audited
purpose of quality control reviews is to enhance entity. The experts provide advice and counsel
the quality of the reports and safeguard against on the drafted preliminary findings, conclusions
insufficient quality of reports. and recommendations and discuss difficult,
ambiguous or contentious issues and alternative
The SAI systematically uses three types of quality reporting strategies. This helps us to improve the
control reviews in all performance audits: peer quality of our reports. The Controller and Auditor
review, review by subject matter experts and General (CAG) personally participates in the
review by three levels of managers. review meetings with the experts. As a
complement to the supervisory review, this
In the peer reviews, colleagues from other means a lot to provide the CAG with assurance
teams review the plans for the audit and the of the quality of the draft report.
draft report. The draft pre-study and the draft
audit report are discussed in meetings with all The supervisors’ review is intended to ensure that
performance auditors. The SAI plans to expand major decisions made by the team and the
the performance audit practice and limit these draft report are reviewed by the senior officials,
review meetings to the different sections. The who can subject the team to rigorous
peer review provides an opportunity for the challenge. Team leaders are supervised by their
audit team to have their judgement tested seniors, Assistant Auditor Generals (AAG) and
against the collective experience and wisdom the Deputy Auditor General (DAG) responsible
of their colleagues. Another benefit is that it for performance auditing. The responsibility of
keeps the reviewers and other staff members the managers is to ensure that audits under their
current with what other teams are doing and jurisdiction are properly conducted according
share innovative approaches (e.g. suitable to the laid down procedures. The SAI recently
techniques for data collection in a certain got the current management structure in place.
locality due to their cultural behaviour) and This structure means that all draft reports will be
successful experiences. This contributes to reviewed by the responsible AAG, as well as by
uniformity and improvement in the performance the DAG and the CAG.
audit practices.
During this process, it is important for reviewers to reconsider carefully the chain of logic that
leads from the raw findings to the analysis and then to the audit conclusions. They need to
consider different perspectives and all relevant arguments before drawing the conclusions. In
particular, it is important that a reasonable reader would reach a similar conclusion from the
same evidence. These kinds of review should be recorded and placed in the audit file. It is
196
important for the audit team to document how they have responded and how any
disagreements were resolved.
How do you consider audited entities or third party comments?
As described in Chapter 6, it is very important that the team discuss the findings with the
audited entities (GUID 3920/129-136). In addition, before finalising the report, it is important
to obtain the audited entities’ and third parties’ views and incorporate any appropriate
changes into the report prior to publication (GUID 3920/136). Some SAIs also publish the
audited entities’ formal comments and an analysis of the comments in the final report for full
transparency.
The Standard
The auditor shall give the audited entity the opportunity to comment on the audit findings, conclusions
and recommendations before the SAI issues its audit report.
When you formally provide the audited entities with a copy of your report, they may provide
comments on:
• the factual accuracy of the report;
• how you have interpreted the facts and draw conclusions; and
• the implications of recommendations you have made.
It is important to examine the audited entities’ response carefully and consider making
appropriate changes to the draft report based on evidence standards. You will already have
discussed your emerging findings with the audited entities, as mentioned above.
Nevertheless, sometimes audited entities respond differently when they have seen your
findings in a report format.
As part of your audit trail, keep a formal summary of how and why you have incorporated any
amendments the audited entities have suggested, as well as a copy of the audited entities’
comments.
The Standard
The auditor shall record the examination of the audited entity’s comments in working papers,
including the reasons for making changes to the audit report or for rejecting comments received.
197
In addition, audit reports often include direct or indirect references to third parties
(organisations, groups and individuals that are not included in the scope of an audit). For
example, your report may refer to a charity or other social grouping representing people who
use public services.
We recommend that you notify such third parties and ask them to verify the accuracy and
completeness of statements concerning them. This process enables the SAI to ensure that
references to third parties are accurate and fair.
How do you publish the final report and communicate the results?
The Standard
The SAI shall make audit reports widely accessible, taking into consideration regulations on
confidential information.
Distributing audit reports widely can promote the credibility of the audit function. Therefore,
audit reports need to be distributed to the audited entities, to the executive and the
legislature, and to other responsible parties. The reports also need to be made accessible to
other stakeholders and the general public directly and through the media, except for reports
that contain sensitive or classified information (ISSAI 3000/134). If you exclude sensitive or
classified content from the published report, you should disclose in the report that certain
information has been omitted, plus give the reasons for the omission. (GUID 3920/138)
SAIs should publish and distribute their reports consistent with their specific mandates (GUID
3920/137). Practices may vary among SAIs. The primary audiences for performance audit
reports is the legislature, executive, government agencies and the public. A good performance
audit enables the legislature to effectively oversee government and agency performance and
influence decision-makers in government and the public service to make changes that lead to
better performance outcomes. However, there are also the general public and other
stakeholders, such as the private sector and the media, who can have an interest, but possibly
a different focus, in the outcome of a performance audit (ISSAI 3000/135). It is important to
communicate to all relevant parties, and you may want to consider generating additional
products. (GUID 3920/141)
198
When writing a performance audit report, remember to ...
… develop a logical and sensible report structure; … make sure the report is independently reviewed
… write recommendations using the SMART to ensure that Supreme Audit Institution standards
are met, and evidence supports the findings and
(Specific, Measurable, Attributable, Relevant and
conclusions; and
Time-bound) framework;
… provide sufficient time to obtain and consider
… ensure that your report is comprehensive,
comments from the audited entity and relevant third
convincing, timely, reader-friendly and balanced;
parties.
199
Chapter 8
How do you follow-up on audit results?

Following-up on audit results
• Determine progress on the audit findings and • Determine financial and non-financial
recommendations. benefits.
• Assess if the problems found were addressed. • Identify areas for future audits.
This chapter discusses the importance of following up on performance audit findings and
recommendations and when to do so.
• What is performance audit follow-up?

• How do you conduct follow-up?
• When do you conduct follow-up?
• How do you determine the impact of the audit?
• How do you report the results of follow-up?
What is performance audit follow-up?
The Standard
The auditor shall follow-up, as appropriate, on previous audit findings and recommendations and the
Supreme Audit Institution shall report to the legislature, if possible, on the conclusions and impacts of
all relevant corrective actions.
The auditor shall focus the follow-up on whether the audited entity has adequately addressed the
problems and remedied the underlying situation after a reasonable period.
Source: ISSAI 3000/136 and ISSAI

3000/139
200
Follow-up refers to your examination of the corrective actions taken by the audited entities,
or another responsible party, on the findings and recommendations of a performance audit.
Follow-up is the last phase of the audit cycle and typically begins after sufficient time has
passed for the findings to be addressed and recommendations implemented. It is an
independent activity that increases the value of the audit process by strengthening the impact
of the audit.
A primary objective of a performance audit is to improve public sector performance and

accountability through the implementation of audit recommendations (see Chapter 1).
Addressing findings and the timely implementation of audit recommendations is the
responsibility of the audited entities. Through a follow-up process, you can monitor whether
and how the problems or findings have been addressed, if the underlying situation has been
remedied, and if the audit recommendations have been implemented by the audited entities.
Remember that it is possible that the auditee has taken other actions to address the finding
rather than implementing the recommendation made. If the intent of the recommendation
is successfully achieved through these actions, the issue should be considered addressed.
According to INTOSAI P-12, reporting on the follow-up measures taken with respect to audit
findings and recommendations is a way to help ensure that those charged with public sector
governance discharge their responsibilities and take appropriate corrective action. Depending
on the SAI’s mandate and wider constitutional arrangements, stakeholders may include the
legislature, its committees and audited entities’ management and governing boards.
In most countries, audited entities are not legally required to implement recommendations
made by SAIs. In addition to providing many benefits, as discussed throughout this handbook,
developing a good relationship with the audited entities can increase the likelihood that it will
address the deficiencies found during the audit and implement the recommendations. During
the audit process and within the report itself, it is important that you provide persuasive
evidence that addressing the findings and implementing the recommendations will bring
considerable benefits to the audited entities, public institutions and the citizens. It is also
important that you follow up on these findings and recommendations to determine whether
they have been implemented and what effects they have had. Follow-up should focus on
whether the audited entity has adequately addressed the deficiencies identified after a
reasonable period of time (ISSAI 3000/140). You as an auditor have to assess in each case
what is a reasonable timeframe for implementation of recommendations, as you cannot
realistically measure results too early following the audit.
Following up on audit findings and recommendations serves several purposes (GUID

3920/146-147):
201
• Identify the extent to which audited entities have implemented changes in response to
audit findings and recommendations. Follow-up can help you determine what actions the
audited entities has taken to remedy any weaknesses identified as a result of the audit.
• Determine the impacts which can be attributed to the audit. The follow-up can reveal
cost savings and non-financial improvements that can be attributed to the audits.
• Identify areas that would be useful to follow up in future work. Following up on findings
and recommendations from previous audits can help the SAI identify cases where it would
be worthwhile to conduct a new audit to determine how performance has changed.
• Evaluate the SAI’s performance. Follow-up provides a basis for assessing and evaluating
SAI performance and may contribute to better knowledge and improved practices in the
SAI. In this respect, following up on audit results can serve as a quality assurance tool.
• Provide feedback to the legislature and government on the impact of the audit. Follow-
up can provide information on the performance and improvements made by the audited
entities in response to the audit.
How do you conduct follow-up?

It is important that SAIs develop a process to follow up on findings and recommendations
made from past performance audits. The audit documentation plays a crucial role in follow-
up because, in many cases, the auditors who conduct the follow-up are not the same as those
who carried out the audit.
When conducting follow-up, it is important for you, as the auditor, to adopt an unbiased and
independent approach for determining whether the audited entity has taken appropriate
actions to address the findings and recommendations. In making this determination, you
should use the same standards and methods used by the team who conducted the
performance audit.
GUID 3920/152 refers to different methods that may be used specifically to follow up on
findings and recommendations. The methods to apply will depend on the procedures and
priorities established by your SAI. Such methods may include the following:
202
• Arrange a meeting with the audited entity after a certain period of Tips on conducting
time has elapsed to find out what actions have been taken in follow-up
response to the audit findings and recommendations. In addition to
After the audit report is
the information gathered during the meeting, the audited entity approved by the Head of
representatives have to provide documentation supporting the the Supreme Audit Institution
(SAI), the audit team leader
corrective actions taken and their effects. could send an action plan
template to the audited
entity to be filled out and
• Request the audited entity to inform the SAI in writing about the returned to SAI. The action
actions it has taken or will take to address the findings and plan should include the types
of actions the audited entity
recommendations presented in the audit report (see Appendix 21 has taken or intends to take
for an example of obtaining actions in writing). to address the findings and
recommendations, as well as
time frames and points of
• Conduct phone calls or limited field visits to collect information on contact.
If needed, the audit team
the actions taken by the audited entity. Both need to be
can meet with
documented. representatives of the
audited entity to discuss and
clarify the information to be
• Keep up to date on reactions from the audited entity and other included in the action plan. It
responsible parties, the legislature and the media to help you is advisable to establish a
deadline for the audited
determine whether problems identified have been appropriately entity to complete and
addressed. return the action plan.
• Request financial and compliance audit teams from your SAI to collect information on the
actions taken in response to your findings and recommendations as part of their audit
procedures and analyse the information and documents received.
• Carry out a new performance audit if needed. The SAI should decide if it is necessary to
conduct a follow-up audit, considering the relevance of the topic and the impact the new
audit might achieve. A follow-up audit could also be a way to evaluate situations when a
problem remains, even when the recommendations have been implemented.
The procedures you use for developing your audit working papers should also be used to
document evidence gathered during follow-up (see Chapter 2 for more information on
organising audit work papers). It is also helpful for you to have a framework for assessing
evidence and determining whether the findings have been addressed and the
recommendations implemented. Appendix 21 contains an illustration that can be used to
conduct this assessment. With such a framework, you can assess and document the extent
and status of implementing your findings and recommendations. When reviewing evidence
on whether the audited entity has fulfilled an audit recommendation, it can also be helpful to
203
have a system for categorising the extent of implementation. For example, you might use the
following categories:
• Fully implemented. The audited entity has taken actions that address the intent of the
recommendation. It is possible that the audited entity addressed the problem with other
actions than those recommended.
• Partially implemented. The audited entity has taken some actions but has not yet fully
implemented the recommendation.
• Not implemented. The audited entity has not taken action to It is important to report
implement the recommendation after sufficient time has passed. the positive action in
responding to the audit
For example, the deadline identified by the audited entity for the recommendations, as this is a
implementation of the recommendation has passed, and the credit to both the audited
entity and the SAI.
audited entity did not address the recommendation.
It can be an extra motivation
for auditors and SAIs to perform
• No longer relevant. A recommendation has been overcome by follow-up activities and can
events or circumstances and is no longer appropriate. have positive impacts on the
SAI’s image, reputation and
credibility.
• Could not be verified. The status of the implementation of the
recommendation could not be determined. As mentioned, Source: IDI/PAS Development Team
sometimes, the follow-up process may reveal significant issues for further review. If
further review is needed, it may be appropriate to carry out a new performance audit. If
your SAI decides to conduct a new audit on the same topic, it is important to determine
why the previous findings and recommendations have not been addressed. In some cases,
other factors may have changed the underlying situation, thus making the
recommendations irrelevant or, for reasons unrelated to the audit, the problem no longer
exists. All of these are considerations for you to make, along with the appropriate timing
for the audit follow-up.
When do you conduct follow-up?
Follow-up is typically done periodically as deemed appropriate by the SAI. The priority of
follow-up tasks is usually assessed as part of the overall SAI´s audit strategy. Sufficient time
has to be allowed to the audited entity to implement appropriate actions. (GUID 3920/148)
Your audit team should begin thinking about follow-up during the course of the audit, and
especially as you are drafting the findings and recommendations. In drafting the
recommendations, as discussed in Chapter 6, your team has to be mindful to ensure they can
204
be appropriately implemented by the audited entity and that the benefit to be derived is
worth the cost to implement them. In addition, toward the end of the audit, it is useful for
your audit team to have some high-level conversations with the audited entity’s senior
management to determine procedures that will be followed for contacting the entity for
follow-up on the findings and recommendations.
For example, the SAI might work out a process with the audited entities. When discussing the
recommendations with the audited entities, it could be helpful to ask them to provide a
timeline for implementing the recommendations. This can be valuable, both for the audited
entity and the SAI, as it can help in planning the follow-up schedule and the actions to be
taken to implement the recommendations. The audited entities can also propose an action
plan.
ISSAI 3000/139 requires the auditor to focus the follow-up on whether the audited entities
have adequately addressed the problems and remedied the underlying situation after a
reasonable period. This reasonable period may depend on the context and nature of audit
recommendations provided. Naturally, some recommendations may require a longer period
to be implemented, while others may require a shorter period. You also have to consider what
type of data can be generated at what time. For example, the effect of the implementation
of the recommendations may only be measured after a sufficient time has passed.
Some findings and recommendations may no longer be applicable. As such, when following
up, you need to concentrate on those that are still relevant. (GUID 3920/151)
The timing of follow-up constitutes a key management decision to be taken by each individual
SAI in accordance with its policies or mandate. For example, the SAI may have a policy of
carrying out follow-up work annually regarding the implementation of audit
recommendations. This practice may help report results systematically, but there may well
be little evidence of impacts in the first year after the publication of the audit report.
Whichever reporting period it chooses, the SAI needs to be clear on any inherent limitations
of its analysis and report accordingly.
For example:
• In SAI Brazil, the follow-up schedule is decided after the analysis of the action plan, which
is completed by the audited entity. Time frames for follow-up are determined according
to the deadlines identified by the audited entities to implement the recommendations.
• In the United States GAO, after conducting and reporting the results of a performance
audit, the auditor's follow-up on the audited entities at least once a year, for four years.
They also measure their effect on improving the government’s accountability, operations
205
and services by tracking the percentage of recommendations implemented within four
years.
• In SAI Georgia, auditors strive to follow up on recommendations twice a year, based on

action plans provided by the audited entities. They are also developing an electronic
recommendation monitoring system to simplify the process. For important audits, they
consider following up on the audits after several years, as appropriate.
• In SAI Philippines, auditors enclose in the transmittal of the performance audit report a
request for the audited entities to prepare an action plan based on the recommendations
embodied in the report. The audited entities complete and provide to the SAI a
standardised action plan form within 60 days of receipt of the report. Follow-up on the
status of implementation of recommendations is undertaken at year-end.
• In the European Court of Auditors and SAI Norway, follow-up normally takes place three
years after the publication of the performance audit report. This allows sufficient time to
pass for the audited entities to implement, if relevant, the audit recommendations.
How do you determine the impact of the audit?
One of the reasons to follow-up is to determine the impact the audit has had on improving
public policies and service delivery. There are different ways to measure the impact of the
implementation of your recommendations. The following examples are adapted from SAI
Brazil and GAO:
• Financial. Benefits related to reductions in expenses or increases in revenues. For example,

the implementation of a recommendation to close a maintenance facility with a low
workload resulted in savings of US $50 million.
• Qualitative and quantifiable. Benefits related to improvements in performance that can

be quantified. For example, the implementation of a recommendation resulted in a 15-day
reduction in the waiting time for lung cancer treatment.
• Qualitative and non-quantifiable. Benefits related to improvement in performance that

cannot be quantified. For example, the implementation of a recommendation resulted in
enhanced safety procedures for personnel handling hazardous materials.
The audit impact has to be considered throughout an audit, from the selection of the audit
topic through audit follow-up. During the follow-up process, the impact of the audited
entities´ implementation of the recommendations can be assessed and measured in different
206
ways. For example, you could compare the situation found during the follow-up with the
situation found during the audit to determine any changes. It is important to separate the
effects caused by the implementation of the recommendation from changes caused by other
factors.
The audited entities may also calculate the impact of the action taken Example of results of
or contract out studies to determine the impact. You could find that audit follow-up
an external organisation has independently evaluated the impact of In 2019, based on follow-up, the
your findings and recommendations. Government Accountability
Office’s (GAO’s) work yielded
US $214.7 billion in financial
For example, in the United Kingdom’s National Audit Office, when an benefits – a return of about US
audit team was following up on the findings and recommendations $338 for every US dollar invested
in GAO. It also identified 1,418
of an audit conducted on major trauma centres, they found that an other benefits – those that
academic study had since been conducted which had measured the cannot be measured in US
dollars but led to programme
impact of the changes made as a result of their audit report. If such and operational improvements
studies exist, you can analyse them and assess whether it is possible across the government.
to use the results as evidence of the impact of the recommendation.
A survey done by EUROSAI has identified six factors that influence audit impact (EUROSAI,
2021). They are:
• Audit report quality.

• Constructive relationship between auditor and audited entities.
• Existence of follow-up system.
• Parliamentary involvement.
• Report the results of the follow-up system.
• Use of the follow-up results for the performance monitoring system and the risk
assessment.
How do you report the results of follow-up?
SAIs may benefit from a system for reporting on the results of follow-up work. Reporting
publicly on the benefits derived from an SAI’s performance audits plays an important role in
showing the value the SAI has brought. This can be helpful for an SAI in justifying their budget
or resource request and can positively enhance their reputation and credibility.
The results of your follow-up efforts may be reported individually or as a consolidated report
which brings together the results of all or portions of your SAI’s follow-up work. Consolidated
207
follow-up reports may include an analysis of common trends and themes across several
reporting areas. Whatever the form, all follow-up reports must be balanced, and findings
presented objectively and fairly. (GUID 3920/155)
A follow-up report could have the following structure:
1. Introduction. Explanation on why the audit was done and information on previous follow-
up activities, if any.
2. Overview. Brief explanation on the audit topic.
3. Methodology. How the follow-up was done.
4. Audit findings. This is the main section of the report. It can contain the findings, the
respective recommendations and the conclusion on the situation found during the follow-
up regarding the implementation of the recommendations.
5. Comments from the audited entities. Summary of the comments made on the draft
follow-up report.
6. Conclusion. Overview of the recommendations´ situation.
Figure 48 has an illustration of an adapted portion of a follow-up report from SAI Brazil of a
performance audit done on a Brazilian programme called ‘Brazil on High-Performance Sports’.
208
Figure 48: Adapted portion of a follow-up report
Excerpt of a follow-up report
… III.4 Socio-educational support to athletes after career

The expression ‘career transition’ is commonly found in the literature to refer to the moment when an
athlete prepares to withdraw from training and competitions. Several reasons can lead to the end of the
career, including the decline in performance due to advancing age, injuries, or even the search for
other occupations in life. This process, therefore, can be planned or compulsory.
In Brazil, the Sectorial Policy for High Performance Sports establishes as one of its objectives to provide
athletes and para-athletes, in the course of their sports careers, the possibility of intellectual and
professional training.
III.4.1 What was reported on the audit
In the audit carried out, it was pointed out that the athlete's post-career theme was not included in the
agenda of actions carried out by the government.
The SAI recommended to the Secretary of High Performance Sports to structure a strategic plan to
reshape the support system to athletes and former athletes, to provide them conditions to stay on sports
area after closing their careers as athletes.
III.4.2 Situation found during follow-up – recommendation not implemented
In the action plan, there is no concrete proposal to implement the recommendation. When asked about
the matter, the audited entity informed there is a project being designed. The objective is to provide
online training to professionals to perform functions related to sport management. That was the only
action mentioned related to the recommendation.
During the follow-up, the SAI conducted a survey, and one conclusion was that the theme is little
addressed in the planning of federations and confederations, with no consolidated strategies aimed at
the socio-educational support of athletes.
With regards to initiatives aimed at supporting the athlete’s education or professionalization in an
alternative career, most leaders of confederations (67%) and federations (57%) who answered the
questionnaire classified their entity’s plans as non-existent or incipient. Regarding the availability of
training programmes, 76% of federations and 67% of confederations pointing out that there is no
planning for this purpose.
When asked about factors that could motivate the desire to abandon their career in the short term, 42%
of the athletes who answered the questionnaire pointed out the item ‘lack of perspectives regarding my
professionalization as an athlete’. This situation ends up influencing the level of satisfaction and
motivation of part of the athletes.
Because of the situation found, the recommendation is considered not implemented.

…
Source: Adapted from a follow-up report from SAI Brazil
An SAI may also report on the results of their follow-up in other ways. For example, the US
GAO maintains a publicly available database of its recommendations and their status. They
use this database, in addition to other mechanisms such as their annual Performance and
Accountability Report, to help communicate the status of their follow-up and the impact of
their work. GAO also publicly reports the percentages of their total recommendations made
within the last four years that have been implemented.
The reporting of follow-up has to be conducted in accordance with the established

procedures of the SAI. Whether or not it is suitable to issue the follow-up audit report to the
209
legislature will depend on how the SAI assesses the significance of the findings, the
conclusions and the impacts of the corrective actions taken.
When following up on a performance audit, remember to ...

… monitor whether your findings and … think about follow-up during the audit, and
recommendations have been addressed; especially as you are drafting the findings and
… report the positive actions taken in responding recommendations;
to the audit recommendations, as this is a credit … assess whether the audited entity’s actions in
to both the audited entity and the Supreme response to the findings and recommendations
Audit Institution (SAI). It can have positive are consistent with the same standards and in the
impacts on the SAI’s image, reputation, budget same manner you assess evidence collected
and credibility; during the audit; and
… adopt an unbiased and independent … document the audited entity’s actions in your
approach for determining whether the audited audit work papers and ensure supervisory review.
entity has taken appropriate actions to address
the findings and recommendations;
210
Appendices
211
Appendix 1: Example of an SAI QA framework for assuring compliance with applicable
standards
Leadership Human Audit Monitoring/

capital performance policy review
Mission, Risk Recruiting/ Performance Audit Referencing Annual Internal

standards management hiring management planning inspections audit
and core
values
Assigning Partner External Quality
Advancement Consultations
staff concurrence audit/audit assurance
with internal
Procedures committee assessments
Tone at and external
for stakeholders
the top
interacting
with Professional
development Peer Professional
audited Supervision practices
Audited entity review
entities and review advisory
comments
and third committee
Procedures party views
Independence for
interacting
with the
Evidence Audit
legislature
documentation
Strategic
planning
Message Public
agreement reporting
212
Appendix 2: Example of a permission to engage in outside activities form
Sample form: Permission to engage in outside activities 1

1. Name: 2. SAI unit/office: 3. Position/title
4. Outside employer, publisher or organisation: 5. Business or activity:
6. Description of your role, work, or product/services you will provide: 7. Does SAI work in this area?
Do you work in this area?
8. Hours to be worked or volunteered? 9. Compensation?
10. Describe business relation, if any, between SAI and outside activity:
11. If a publication or speech, describe subject and list any related SAI products:
Note: The text of any outside publication or prepared speech must be reviewed by the appropriate unit or office
before publication or delivery.
12. As provided in SAI Order 123, I request SAI's permission to engage in the outside activity described above. I will
not engage in the activity during my hours of official duty.
Auditor signature and date:
13. Supervisor's recommendation
____ Approve ____Approve with conditions (attach summary of conditions) ____Disapprove (attach summary of
reasons)
Signature of supervisor:
14. Signature of approving official
Approval expires in 3 years, unless earlier date is entered here:
1SAI employees can use this form to request permission to engage in outside activities, which may be required before:
(1) engaging in an activity for which compensation, salary or fee is received in exchange for the individual's personal
time, effort or talent (excluding reimbursement for travel or other expenses actually incurred in performing the activity
or employment); (2) engaging in an activity for which compensation, salary or fee is customarily received, even if the
employee performs the activity gratuitously; (3) speaking or writing, even if the outside speaking is or is customarily
performed gratuitously; (4) serving as an officer, director, trustee or spokesperson for an association or organisation;
and (5) running for elective office, where permitted by SAI order.
Source: US GAO
213
Appendix 3: Example of an independence statement
Sample form: Statement of auditor independence
In all matters relating to SAI's audits, the SAI, its employees, and others involved must be free from
circumstances that would cause a reasonable and informed third party to doubt their integrity, objectivity or
professional sceptic ism, and must maintain independence of mind and in appearance.
Threats to Independence include, but are not limited to,
a. Self-interest threat - the threat that a financial or other interest will inappropriately influence an
auditor's/investigator's judgement or behaviour.
b.Self-review threat - the threat that an auditor or audit organisation that has provided nonaudit services will
not appropriately evaluate the results of previous judgements made or services performed as part of the
nonaudit services when forming a judgement significant to an audit.
c. Bias threat - the threat that an auditor will, as a result of political, ideological, social or other convictions,
take a position that is not objective.
d.Familiarity threat - the threat that aspects of a relationship with management or personnel of an audited
entity, such as a close or long relationship, or that of an immediate or close family member, will lead an
auditor to take a position that is not objective.
e.Undue influence threat the threat that external influences or pressures will impact an auditor's ability to
make independent and objective judgements.
f. Management participation threat the threat that results from an auditor's taking on the role of management
or otherwise performing management functions on behalf of the entity undergoing an audit
g. Structural threat the threat that an audit organisation's placement within a government entity, in
combination with the structure of the audited entity will impact the audit organization's ability to perform
work and report results objectively.
Completing this form underscores the importance to adhere to standards of independence and objectivity, and
must be annually certified by all SAI employees involved in audits. Individuals who are unable to make this
certification or believe a threat to their independence that could require safeguards may exist must notify a
senior manager involved in their current assignment to discuss their situation.
I certify that there are no impairments to my independence, and I will promptly notify a senior manager on
my current assignment if a threat to my independence that may require safeguards should arise.
Signature: Date:
214
Appendix 4: Example of an audit topic selection matrix
Audit topic selection matrix

The SAI identified four possible audit topics via its strategic planning process:
1. Solid waste management
2. Climate change adaptation
3. Sustainable fisheries
4. Maternity services in public hospitals
The following table illustrates an example of how scoring is assigned, and audit topics prioritised based on
selected criteria. (The topic selection criteria and their weights will be chosen in accordance with the ISSAI
and based on their relevance and importance to the SAI).
CRITERIA WEIGHTS Identified alternative audit topics

Topic 1 Topic 2 Topic 3 Topic 4
Score Weighted Score Weighted Score Weighted Score Weighted
score score score score
1. Materiality 15 3 45 3 45 2 30 2 30
2. Auditability 15 3 45 1 15 2 30 2 30
3. Possible impact 15 3 45 2 30 2 30 2 30
4. Risks to the SAI 10 3 30 1 10 3 30 3 30
5. Legislative or 10 3 30 3 30 0 0 3 30
public interest
6. Relevance 10 3 30 3 30 2 20 3 30
7. Timeliness 5 3 15 3 15 1 5 2 10
8. Previous audit 5 2 10 3 15 2 10 2 10
work
9. Other works 5 2 10 1 5 1 5 1 5
planned or in
progress
10. Request for 10 3 30 0 0 3 30 2 20
performance
audit
Aggregate 100 290 195 190 225
weighted score
Rank 1 3 4 2
Note: Not applicable = 0, Low = 1, Medium = 2, High = 3
Comments:
1. Materiality:……………..(state here the reasons/justifications for specific score for each topic)
2. Possible impact:………………………………………………………………………………………………..
3. Relevance:………………………………………………………………………………………………………
4. ……………………………………………………………………………………………………………………..
The above assessment indicates Topic 1 (Solid waste management) as the first priority, Topic 4 (Maternity services in
public hospitals) as the second priority, Topic 2 (Climate change adaptation) as the third priority and Topic 3
(Sustainable fisheries) as the fourth priority.
✓ Weights are assigned to each criterion and aggregate to 100%. The assignment of weight to each criterion will
depend on the importance of the criterion to the SAI’s management, the legislature, the government and the
public in general.
✓ The auditor should exercise professional judgement while assigning a score of ‘not applicable’, ‘low’, ‘medium’ and
‘high’. However, their judgement should be backed with appropriate justifications and documentation.
✓ The product of ‘weights’ and ‘score’ would give the ‘weighted score’. The aggregate of weighted score would
result to ‘aggregate weighted score’ for each topic.
✓ The topic scoring highest ‘aggregate weighted score’ can be ranked as the first priority. Hence, it
would generally be accorded the highest priority for audit resources, and subsequently be prioritised for
audit. The numbers of audit topics chosen to be audited in a given period will depend on the
availability of audit resources. The topics chosen will be based on priority determined through rank
combined with professional judgement.
Note: This example uses weighted scores, but SAIs may choose not to use weights if all criteria seem to be of equal importance.
215
Appendix 5: Sample design documents
Stakeholder analysis
During the pre-study, it is critical that you work with stakeholders inside and outside your
SAI. Examples of internal stakeholders are methodologists or legal experts. Examples of
external experts are subject matter experts that specialise in the subject of the audit. To do
this effectively, you may find it beneficial to complete a stakeholder analysis. Figure 5.1
provides an example of an analysis of stakeholders completed as part of an audit examining
issues of domestic violence and violence against women.
216
Figure 5.1: Sample stakeholder analysis for a performance audit
Stakeholder Roles Interests Priority

for the
audit
Victim/survivor • Report physical, psychological, sexual, • Receive proper care High
patrimonial and moral aggression. and treatment.
• Request support and shelter (if needed). • Feel safe.
• Be aware of procedural acts concerning • Go back to normal
the offender. activities.
• Don’t suffer
violence.
• Know that the
perpetrator will be
punished.
Perpetrator • Seek help to stop being violent. • Receive proper care High
and treatment.
• Change behaviour
and attitudes.
Children/family/dependents • Report physical, psychological, sexual, • Receive proper care Medium
patrimonial and /or moral aggression. and treatment.
• Feel safe.
• Go back to normal
activities.
Centre of government • Coordinate and integrate the policies of • Implementation of High
multiple ministries/departments. the nationally
• Set out plans to address for SDGs agreed targets
implementation. linked to the SDGs.
• Review and refine implementation of
policies linked to SDGs.
• Assess how well policies are being
implemented.
• Provide information.
• Ensure inclusiveness in implementation
plans to ‘leave no one behind’.
Ministry of Women • Formulate and coordinate policies for • Decrease of violence High
prevention and protection of women against women in
victims of violence. the country.
• Prepare national plan on gender equality.
• Promote gender equality.
• Develop and implement awareness-
raising campaigns about violence against
women.
• Articulate, promote and implement
cooperation initiatives with national and
international public and private entities to
help the implementation of policies for
women.
Regional/local gov. • Implement plans on gender equality. • Decrease of violence High
institutions responsible for • Promote gender equality. against women in
actions of EIPV • Develop and implement awareness- their area.
raising campaigns about violence against
women.
217
Ministry of Health • Establish rules, guidelines and protocols • Ensure to victims of High
for care of victims of IPV. IPV all the necessary
• Provide multidisciplinary teams (nurses, support for the
doctors, psychologist, social workers) to restoration of their
care for victims of IPV. health.
• Prevent sexually transmitted diseases to
victims of IPV.
• Provide services for legal abortion in
cases of IPV.
• Support technically and financially the
organisations responsible for EIPV.
Ministry of Justice • Establish policies and plans to provide • Good service High
the necessary services to those impacted provided to those
by IPV (victims, perpetrators, families). impacted by IPV.
• Coordinate the implementation of
policies and plans among the institutions
responsible for EIPV (police stations,
legal system, judges, public prosecutors,
district attorneys).
Ministry of Education • Promote educational campaigns to raise • Successful education High
awareness against IPV. activities to
• Review school curriculum to ensure that decrease IPV.
they are free from gender stereotypes.
• Develop capacity programmes for
teachers and other professionals
responsible for education focusing on
gender equality and EIPV.
Ministry of Social Welfare • Establish policies and plans to provide • Ensure the welfare High
the necessary services to those impacted of victims and their
by IPV (victims, perpetrators, families). families.
• Support technically and financially the
organisations responsible for EIPV.
• Coordinate the implementation of
policies and plans among the institutions
responsible for providing services to
those impacted by IPV.
Police Department • Ensure police protection to the victim, if • Provide good High
needed. services to victims.
• Refer the victim to the hospital, if • Contribute to EIPV.
needed.
• Refer the victim to the prosecutor, if she
wants to press charges against the
perpetrator.
• Request protective measures from the
judge, if needed.
National Statistical Office • Develop and maintain a data system for • Provide reliable and High
collect, compile and analyse data on IPV. good quality
• Receive and compile data about IPV statistical
received from states and municipalities. information about
• Assess the integrity of data received. IPV.
• Develop and communicate reports with
statistical information about IPV.
CSOs that work with EIPV • Mobilise society on the issue of IPV. • Ensure the welfare Medium
of victims of IPV.
218
• Claim actions and measure to improve
care for victims of IPV and their children.
• Inform and educate victims about their
rights.
• Assist victims of IPV and their children in
the areas of education, physical and
mental health, employment, housing,
access to justice.
UN agencies • Mobilise governments and society on the • Ensure women Medium
issue of IPV. rights.
• Claim actions and measure to improve • Decrease of violence
care for victims of IPV and their children. against women.
rights.
Experts • Conduct studies and researches on EIPV. • Decrease of violence High
• Provide qualified information to against women.
governments and CSOs on IPV.
• Support government agencies in
formulating and implementing policies
on EIPV.
Women’s association • Mobilise society on the issue of IPV. • Ensure welfare and Medium
(national, province, • Claim actions and measure to improve safety of victims of
municipality, village) care for victims of IPV and their children. IPV.
rights.
• Assist victims of IPV and their children in
the areas of education, physical and
mental health, employment, housing,
access to justice.
Judges • Grant protective measures. • Ensure welfare and High
• Inform prosecutor about requirement of safety of victims of
protective measures. IPV.
• Order the perpetrator’s custody, if
needed.
• Revoke custody, if applicable.
Public prosecutors • Request police protection for victims of • Ensure welfare and Medium
IPV. safety of victims of
• Request health, education, social welfare IPV.
and other services for victims of IPV.
• Supervise public and private
establishments that provide the
necessary services to those impacted by
IPV (victims, perpetrators, families).
District attorneys • Provide specific and humanised legal • Allow access to Medium
service to victims of IPV. justice for victims of
IPV.
Source: IDI’s SDGs Audit Model (ISAM)
Initial meeting agenda
219
Some SAIs find that it is helpful to gather internal stakeholders (for example, legal experts,
economists, individuals with technical expertise) to participate in an initial meeting at the
beginning of the performance audit. During this meeting, you will discuss with your
stakeholders possible approaches, audit questions, design options and potential points of
contact who have knowledge of the audit topic. The following is a sample agenda from GAO
that can guide this type of meeting.
220
Sample initial meeting agenda
SAI management Audit team Stakeholders

Group 1 Juan Baldéz, Auditor in Charge General Counsel
Sara Peck, Managing Director Alessandra Engle, Auditor Rebeca Sanchez, Legal Expert
Maita Subramanian, Director
Ling Liu, Audit Manager Methodologist and Data Analysis
Support
Melissa Ngumo (methodologist)
Takano Watanabe (data analyst)
Discussion of audit (5 minutes)

Basis for the audit (law, request or other).
Estimated resource requirements.
Risks for weaknesses in performance.
Challenges in meeting the principles of economy, efficiency and effectiveness, including being able to comply
with laws and regulations.
Internal controls.
Internal/external coordination (10 minutes)
Summary of meetings with internal stakeholders and audit teams working on related topics or with the
audited entity.
Summary of meetings with external stakeholders, including other audit organisations, research groups or
those who have examined the topic of the audit.
Known ongoing activities at the audited entity pertaining to the substance of the audit.
Planning to schedule meeting with relevant subject matter experts.
Proposed audit objectives (15 minutes)
What have been the trends in … ?
To what extent does the division … ?
How consistently and adequately is the division … ?
Potential methodologies (15 mins)
Data analysis: Analyse data from the … fiscal years.
Interviews with knowledgeable officials at … offices.
Site visits at … locations.
Discussion of stakeholder roles (10 minutes)
Recap of key decisions made at the meeting and post-meeting action items (5 minutes)
Document decisions.
Hold initial meeting with audited entity.
Source: US GAO
Agenda for initial meeting with the audited entity
Audit teams typically meet with the audited entity prior to starting information and data
collection. During this meeting, teams meet with officials to introduce their work and identify
221
their information needs for the audit, among other things. The following is a sample agenda
from the GAO.
Sample agenda for initial meeting with the audited entity
Date/time:
Location:
Dial-in information:
Attendees
Name Title Email Phone

Joan Smith Director
Ling Liu Audit Manager
Juan Baldez Auditor-in-Charge
Alessandra Engle Auditor
Source of work: (for example, request from oversight committee, part of the SAI’s ongoing
audit topic, etc.)
Scope of work: This work includes efforts to assess the management of the government’s
[describe audit objective and audit questions.] As this audit proceeds, our information needs
may expand, and additional information may be needed. We will inform you of these changes
as they are identified.
Offices and locations that the SAI has initially identified to conduct work include:
Time frames
• We plan to begin our work immediately and seek to have a draft report completed by …
20XX.
Source: US GAO
Tools for enhancing subject matter knowledge and analysing risk
SWOT (Strengths, Weaknesses, Opportunities and Threats) analysis

The purpose of the SWOT analysis is to identify and categorise strengths, weaknesses,
opportunities and threats associated with the internal and external environments of the
audited entity. Figure 5.2 shows the SWOT template, while Figure 5.3 provides a sample
SWOT analysis.
222
Figure 5.2: SWOT template
Internal environment External environment

Strengths Opportunities
Positive
Identify and list strengths related to Identify and list opportunities that
the internal environment of the exist in the external environment of
audited entity. the audited entity.
Weaknesses Threats
Negative
Identify and list strengths related to Identify and list threats that exist in
the internal environment of the the external environment of the
audited entity. audited entity.
223
Figure 5.3: Sample SWOT analysis for a performance audit on a rural drinking water supply
scheme
Internal environment External environment
Strengths Opportunities
Positive
1. Clear, established goal related to 1. River runs across the length of the country
Sustainable Development Goals 2. Established criteria from World Health
2. Rural drinking water supply scheme Organization for testing water quality
exists since 2001 3. Increase in tourism brings more financial
3. Entities responsible for water supply on resources to the country
three government levels
4. Well-defined responsibilities among
three government levels
5. Annual plans defined
6. Monitoring cell
7. Existence of Village Water and
Sanitation Committee
8. Transparent process for contracts to
establish water supply schemes
Negative
Weaknesses Opportunities
1. Behind established goal (26% still do not 1. Biological surface water contamination
have access to safe drinking water) 2. Chemical groundwater contamination
2. Non-availability of drinking water all the 3. Little articulation between water supply
time in some villages programme, sanitation service and
3. Shortage of testing laboratories (should health system
be one per district) 4. Little coordination among districts and
4. Shortage of material for testing kits villages to share water
5. Few people training to use test kits 5. Dependency on weather conditions
(should be five per village) 6. Increased tourism can lead to a rise in
6. Shortage of people to maintain water demand for water
distribution network in order to avoid
water leakage
7. Shortage of people in districts and
villages to regularly inspect the water
structures and to operate and maintain
water supply schemes
8. Shortage of village financial resources for
regular maintenance
9. Some districts have less water coverage
10. No gender equality in composition of
Village Water and Sanitation Committee
11. Deficiencies in water storage
12. Lack of water harvesting (collect from
rain)
13. Lack of information system and
performance indicators
Risk verification diagram (RVD)
To develop the RVD, make a list of the risks associated with the weaknesses (w) and
threats (t) you identified in the SWOT analysis, evaluate them for probability and impact,
224
and then place them accordingly in the diagram. Figure 5.4 shows a sample RVD based on
the rural drinking water supply scheme SWOT analysis above.
Figure 5.4: Sample RVD for audit of a rural drinking water supply scheme
Low probability/ Medium probability/ High probability/

high impact high impact high impact
Difficulty of having the Water leakage (W6, W7, W8) Not reaching the Sustainable
annual plan approved (W10) Development Goal (W1,
Low priority of the subject in W13, T1, T2, T5)
municipality (W10) Increase in disease due to
contaminated water (W1,
Not taking advantage of the W2, W3, T1, T2, T3, T5)
possibility to save money Bad quality water (W3, W4,
(W12) W5, W6, W7, W8, W9, T1, T2)
Waste of public resources
Management difficulties (W6, W7, W8, W11, T3)
(W13) Less water testing (W10)
Inefficiency (W12)
Difficulty of measuring
performance (W13)
Increase in expenses with
water treatment (T1, T2)

medium impact medium impact medium impact
Biased view of water Rural exodus (W1, T1, T2, T5) Water storage in bad
problems (W10) Low frequency of Village conditions (unclean tanks,
Water and Sanitation bowls, buckets) (W2)
Committee meetings (W10) Difficulty of writing annual
plan (W13)

low impact low impact low impact
Schools and offices have to Increase in health expenses

Impact
be closed (W2) (W1, W2, W3, T1, T2, T3, T5)

Women less empowered
(W10)
Probability
225
Appendix 6: Design paper checklist
The design paper could be used as an alternative to the design matrix. It allows audit teams
to document their audit’s design in narrative form outside a structured matrix. The design
paper can take multiple forms, depending on the audit’s circumstances and team or
management preferences. Figure 6.1 can help ensure your design paper includes the
necessary information.
Figure 6.1: Design Paper Checklist
Checklist for design paper

(a) Does the design paper identify either: i) the criteria to be used to evaluate the matters
you are auditing; or ii) the planning to be undertaken to identify the criteria needed to
evaluate the matters you are auditing?
Examples of possible criteria include: the purpose, goals, policies or procedures prescribed
by law or regulation or set by management; technically developed standards or norms;
expert opinions; prior years’ performance; performance of similar entities; performance in
the private sector; or best practices of leading organisations.
(b) Does the design paper include sufficient information to provide context for the audit
(for example, the nature of the issue, the significance of the programme, the potential
problem or concern and its magnitude, the political environment, key players and
potential users of the audit product)?
(c) Does the design paper identify the audit questions?
(d) Does the design paper identify the sources for the information needed to answer the
audit questions and where that information will be obtained or how you plan to identify
potential sources of data that could be used as audit evidence?
(e) Does the design paper identify how you are going to follow-up on known significant
findings and open recommendations identified in previous audit reports that relate to the
audit’s objectives?
(f) Does the design paper include the overall design strategy or methodology for
answering the audit questions and the types of analysis to be used? Methodologies could
include case studies, structured interviews, focus groups, file reviews, visual inspections,
sampling or use of computer-based data.
(g) Does the design paper document the limitations to the work (for example, difficulty
gaining access to records, staffing and travel constraints, or data quality or reliability
issues) and their effect on the product?
(h) Does the design paper include what you expect the analysis will allow you to say?
Source: US GAO
226
Appendix 7: Project schedules and work breakdown structure
Basic project schedule
To complete the basic project schedule, enter tasks and milestones in their general order of
occurrence. Tasks may be undertaken concurrently. For each task, identify the start and end
dates, along with the audit team member(s) assigned. The project schedule is structured to
group tasks by phase, in alignment with the Supreme Auditing Institution’s (SAI) audit
process. Figure 7.1 shows a sample basic project schedule. The sample is abbreviated for
illustration purposes; the number of tasks and milestones have to be modified to fit the audit
plan.
Figure 7.1: Sample basic project schedule
Paragraph START END

STAFF ASSIGNED
DATE DATE
Phase 2 – Designing the Audit (structure tasks around SAI internal audit process)
Task 1 (e.g., conduct pre-study of audit topic)
Task 2 (e.g., develop proposed audit objective(s), scope, and

methodology – and identify criteria)
Task 3 (e.g., assess design risk)
Task 4 (e.g., prepare audit plan)
Phase 2 Milestone (e.g., reach consensus on audit plan)
Phase 3 – Conducting the Audit (structure tasks around SAI internal audit process)
Task 1 (e.g., gather evidence)
Task 2 (e.g., analyze evidence)
Task 3 (e.g., evaluate evidence for sufficiency and

appropriateness)
Task 4 (e.g., develop audit finding)
Phase 3 Milestone (e.g., reach consensus on report message)
Phase 4 – Reporting (structure tasks around SAI internal audit process)
Task 1 (e.g., draft report)
Task 2 (e.g., verify facts and obtain audited entity comments)
Task 3 (e.g., finalise and obtain SAI management approval)
Phase 4 Milestone (e.g., disseminate report)
Source: US GAO
227
When building your project schedule, remember that the plan must be realistic to effectively
guide the audit process. Planning the sequence and duration of activities can be challenging,
particularly as the audit unfolds and new information or factors emerge. In addition, auditors
frequently spend time on non-audit activities, such as other SAI responsibilities, training and
holiday. As a result, it is common for auditors to be overly optimistic when estimating the
duration of the audit and its key activities, such as information gathering, analysis, report
writing and review. The audit risks you identified (see Chapter 4) will provide an added layer
of ambiguity that must be accounted for when you allocate to related tasks. For these
reasons, it is helpful to avoid being overly detailed when developing your schedule. This will
help to limit the time you spend modifying the plan as the audit matures.
Detailed project schedule
A detailed project schedule allows you to closely define and link the work, task
dependencies, durations and resources. While you can create and manage a basic project
schedule on paper or using basic software applications, a detailed project schedule is more
easily managed using project management software, which may be purchased or found open
source.
Like the basic project schedule, you complete the detailed schedule by entering tasks and
milestones in their general order of occurrence. For each task, identify the duration,
resources and any task dependencies by sequentially linking tasks. For example, if an
interview must be conducted before completing an analysis, the interview will be linked to
the analysis as a predecessor task. However, tasks will often run concurrently or overlap to
varying degrees.
While you may enter specific dates for an activity that must occur at a precise time, it is
generally preferable to allow the audit duration and activity dates to be shaped by each
activity’s duration and dependencies, including the predecessor and successor activities is
linked. When adding resources, the detailed schedule also allows for the allocation of specific
hours, which may aid in more accurately determining the workload associated with tasks.
The sample shown at Figure 7.2 is abridged to show the possible detail and sequencing of
activities in the planning phase only. You can replicate this model for all other audit phases
to build a comprehensive project schedule.
228
Figure 7.2: Sample detailed project schedule for the planning phase
Name Duration Start Finish Predeces Resources

(days) sor task
1 Audit Name
2 Phase 2 - Planning 29 1/2/2020 2/12/2020
3 Conduct pre-study and consider audit approach 11 1/2/2020 2/17/2020
4 Review previous work on the audit topic and 5 1/2/2020 1/9/2020 Auditor 1;Auditor 2
perform background research
5 Discuss the topic with the audited entity 3 1/9/2020 1/14/2020 4 Auditor 1; Auditor 2; Audit Mgr
6 Identify and engage with internal stakeholders 8 1/2/2020 1/14/2020 4 same Auditor 1; Auditor 2; Audit Mgr
start (ss)
7 Determine audit approach 3 1/14/2020 1/17/2020 5 Auditor 1; Auditor 2; Audit Mgr
8 Develop objectives, scope and methodology 13 1/17/2020 2/5/2020
9 Determine scope and objectives of audit 1 1/17/2020 1/18/2020 3 Auditor 1; Auditor 2; Audit Mgr
10 Determine audit questions 2 1/20/2020 1/22/2020 9 Auditor 1; Auditor 2; Audit Mgr
11 Identify audit criteria 3 1/22/2020 1/27/2020 10 Auditor 1; Auditor 2; Audit Mgr
12 Determine time frames and resource needs 5 1/17/2020 1/24/2020 9ss Auditor 1; Auditor 2; Audit Mgr
13 Determine audit methodology 7 1/27/2020 2/5/2020 10;11 Auditor 1; Auditor 2; Audit Mgr
14 Assess design risk 5 1/22/2020 1/29/2020
15 Prepare SWOT analysis and RVD 2 1/23/2020 1/27/2020 13ss- Auditor 1;Auditor 2
2days
16 Determine risk tolerance 2 1/22/2020 1/24/2020 15ss-1 Auditor 1; Auditor 2; Audit Mgr
day
17 Identify steps to mitigate design risk 2 1/27/2020 1/29/2020 15;16 Auditor 1; Auditor 2; Audit Mgr
18 Prepare audit plan 11 1/28/2020 2/12/2020
19 Prepare design matrix 10 1/28/2020 2/11/2020 10;13ss+ Auditor 1;Auditor 2
1day
20 Prepare basic or detailed project schedule 5 1/28/2020 2/4/2020 19ss Auditor 1;Auditor 2
21 Prepare work breakdown structure (optional) 5 1/28/2020 2/4/2020 19ss Auditor 1;Auditor 2
22 Obtain management approval of audit plan 1 2/11/2020 2/12/2020 3;8;14;19 Auditor 1; Auditor 2; Audit
;20;21 Mgr; SAI mgmt
While not required, one benefit of a detailed project schedule is that it will enable you to more
easily determine the critical path. As discussed in Chapter 4, the critical path is the path of
longest duration through the sequence of activities in your schedule. Establishing the critical
path determines the audit’s earliest completion date and allows the team and management
to focus attention on the activities that could cause audit timelines to slip. Accordingly, it is
generally preferable to include the audit’s most important activities on the critical path.
Work breakdown structure
A work breakdown structure is often best used when trying to define the various specific
tasks associated with a certain method, such as a survey. It can be developed using basic
word processing applications or project management software. To develop the work
breakdown structure, create a hierarchical tree structure starting with the main task. You
will then subdivide the main task into subordinate tasks, which should in total constitute
229
fulfilment of the main or ‘parent’ task. Tasks can be subdivided to the extent necessary and
reasonable, culminating in the terminal task – which is the last task that is not subdivided.
Responsible parties could also be associated with each task to clearly define who is
performing the work.
Figure 7.3 shows a simplified work breakdown structure for conducting a survey. Additional
tasks and subtasks can be added to this structure at each level to achieve the desired level
of detail. Further, if desired, a smaller work breakdown structure could be placed within a
broader one covering the entire design phase or entire audit.
Figure 7.3: Work breakdown structure sample
1.0 Design 2.0 Administer 3.0 Analyse

survey survey survey
(team/specialists) (team/specialists) (team/specialists)
1.1 Obtain information 2.1 Make advance 3.1 Process data

needed to design contacts with survey (specialists)
survey (team) recipients (team)
3.2 Analyse and

1.2 Determine survey 2.2 Launch the survey summarise data
population and (team/specialists) (team)
sample
(team/specialists)
2.3 Review preliminary
results and conduct
1.3 Develop and test follow-up with non-
survey questions respondents (team)
(team/specialists)
1.4 Select survey

method (for example,
web, email, phone)
(team/specialists)
While these are optional, work breakdown structures can help you better define the
scope of effort for a major method and break the work into smaller, more manageable
components. By doing so, you may also enable the audit team to more accurately identify
and tally costs and labour hours associated with the method.
230
Appendix 8: Interview guide
Planning the interview

Step 1: Complete pre-interview research:
1. Identify purpose and goals.
2. Develop sufficient background.
3. Identify who will be interviewed.
4. Identify other sources of information needed.
Step 2: Prepare questions:

1. Determine what you want to know.
2. Draft questions.
3. Have your supervisor review the questions you have drafted.
Step 3: Prepare logistics:

1. Schedule the time, place and location of the interview.
2. Inform the person being interviewed about the purpose and goals of the interview.
3. Decide how many staff will attend. Try to have more than one interviewer attend.
Conducting the interview

Step 4: Open the interview:
1. Be punctual and dress appropriately.
2. Consider conducting some small talk, if appropriate, to put officials at ease.
3. Provide introductions, purpose of the interview and background on the audit.
4. Explain desired outcomes.
Step 5: Conduct the interview:

1. Ask the questions you have prepared.
2. Practice active listening.
3. Ask probing questions:
• Don’t just accept statements at face value – ask for elaboration and supporting
documentation.
• Ask what the problems are, why they exist and how the people interviewed would change
the audit topic; ask who, what, where, when, how and why.
4. Ask for reasons and examples to support the information provided.
5. Be prepared to ask follow-up questions that may not be on your predetermined list of
questions.
6. Follow new lines of enquiry when topics or responses are presented that you did not
anticipate.
7. Clarify ambiguous responses.
8. Ask people being interviewed to spell out any acronyms with which you are not familiar.
231
9. Ask for definitions of key terms and technical jargon.
10. Take detailed notes of the responses to your questions.
11. Maintain a list of documents to be obtained.
12. Maintain control of the interview.
13. Focus the interview on relevant information.
14. Watch for topics that officials try to evade.
15. Respect time limits.
16. Ask for related documentation and referrals.
Step 6: Close the interview:

1. Summarise key information obtained.
2. Explain how the information will be used.
3. Address any questions or comments from the person interviewed.
4. Ask if it would be appropriate to contact the person interviewed with any follow-up
questions.
5. Thank the people interviewed for their time and information.
Debriefing the interview

Step 7: Debrief the interview:
1. Did you accomplish what you set out to do? If not, why not?
2. What could you have done differently?
3. Where does the audit team need to go from here?
4. Did everyone on the audit team hear the same information?
Step 8: Complete post-interview activities:

1. Write up the interview record as soon as possible after the meeting (see additional
guidance below).
2. Send the draft interview record to your supervisor for review.
3. Obtain identified documentation from the people you interviewed.
4. Schedule follow-up interviews as needed.
Documenting the interview

The purpose of documenting the interview is to: document the facts of what was said in the
interview and by whom; and organise these facts to help you develop findings.
1. Prepare the interview record as soon as possible after the meeting to document the
testimonial evidence obtained as completely and accurately as possible:
• Use your notes and the notes from team members who also attended the meeting to
record it as accurately as possible.
• Generally, it is useful for interview records to be organised logically by topic, preferably
with the most important material being presented first. Keep in mind that while the record
is to be as detailed as possible, it is not a transcript of the interview.
232
• Cross-reference all documents referred to in the record.
• Resolve all open remarks or unanswered questions:
o Use auditor notes to help explain context, circumstances, prejudice or other
contributing factors to the interviewees’ statements.
o Define all acronyms and abbreviations when they are first discussed.
2. Keep the audit objective(s) and questions in mind as you prepare the record:
• Assess whether you are gathering the data you need to address the audit questions.
• Use headings/sub-headings in the record to organise the information whenever possible.
• If necessary, ask your supervisor if it would be useful for you to confirm any information
you gathered during the interview.
3. Ask your supervisor if it would be useful for other team members who attended the
interview to review your document for accuracy.
4. Provide the draft interview record to your supervisor for review.
Additional guidance
Interviewing is both a data-gathering tool and a data-analysis tool. When you conduct an
interview, you are gathering evidence to support potential findings.
Before the interview
Consider a sequencing strategy for your questions. Although there is no particular sequencing
structure for conducting interviews, it may be helpful to anticipate how you will use the
information you gather during the interview. The answer to this question may lead you to
decide how the interview will be structured. The following are examples you might wish to
consider.
Funnel sequence. Begin with the most general questions and then narrow the focus and
become more specific with each succeeding question. This method provides more specificity
and clarity to general answers that are initially provided. This method may cause the person
being interviewed to revise initial statements to provide accuracy.
Inverted funnel sequence. Begin with the specific questions and conclude with the most
general questions. This method can help the interviewer develop relationships between the
specific issues being discussed and other issues that may be important to the study.
Sensitivity sequence. Consider placing the most difficult or sensitive questions at the end of
the interview. This method will help the interviewer maintain an open flow of communication
for as long as possible. An alternative is to acknowledge at the beginning of the interview with
the person being interviewed that you have a sensitive issue to discuss and decide whether
to begin or end the interview with the sensitive issue.
Chronological sequence. Start with the beginning of a process or timeline and follow it
through in the order of events. This method is particularly helpful during interviews at the
beginning of an assignment when the interviewer is obtaining background information.
233
Random sequence. No particular order may be needed if all the questions have equal
importance.
During the interview

Practice active listening:
• Suppress disruptive behaviour (finger drumming, pencil tapping, fidgeting).
• Do not gaze out of the window or read diplomas or certificates on the wall.
• Do not begin reading documents you are given while the official is speaking.
• Do not let your biases or knowledge obtained elsewhere interfere with the message from
the person being interviewed. Keep an open mind.
• Do not jump to conclusions; listen to the person being interviewed. As much as you may
be tempted to develop a finding, do not put words into the official’s mouth.
• Do not interrupt or debate.
• Do not assume what the person being interviewed meant. Request clarification. Do not
monopolise the conversation or try to have the last word.
• Be prepared to adjust your planned set of questions if necessary. However, do not jump
ahead. Concentrate on what the official is saying at the moment.
• On key points, summarise or repeat back in your own words what you believe the person
being interviewed has just said. Give the official the opportunity to make corrections.
• Show the person being interviewed that you are listening.
• Try to motivate the person being interviewed to communicate more fully.
Avoid common pitfalls:

• The interviewer uses lots of words but never gets to the point. The person being
interviewed never really hears a question and, therefore, cannot really provide an effective
answer.
• The interviewer asks multiple questions in one. The person being interviewed is not sure
which question to answer. In other cases, the person being interviewed answers one part
of the question, but the other parts are lost.
• The interviewer asks a ‘yes/no’ question when an open-ended question may be more
appropriate.
• The interviewer asks leading questions by identifying the expected answer in the question
or by using emotionally-loaded words.
234
Example of a record of interview
Title Meeting with audited entity
Purpose Gather information about … (details about audit topic)
Contact method In-person
Contact place Physical or mailing address of meeting
Contact date (insert date)
Audited entity:
Participants Jane Doe, title, phone number, email address
John Doe, title, phone number, email address
Supreme Audit Institution:

Audit team member name, title, phone number, email address
Audit team member name, title, phone number, email address
Comments/remarks:
We interviewed Jane Doe and John Doe during our site visit to their facility. We asked them questions
about their audit entity’s participation in the audit topic.
Jane Doe gave a description of the audit entity’s relationship to the audit topic. The relationship is: …
She also discussed how long the audit entity had been participating in the audit topic, which is …
amount of time. Jane Doe also described her role and responsibilities at the audit entity, as well as
how her roles and responsibilities related to the audit topic. Her role is … and responsibilities are …
and … . They are related to the audit topic because … . John Doe also shared his role and responsibilities
at the audit entity, as well as how his roles and responsibilities related to the audit topic. His role is …
and his responsibilities are … and … . They are related to the audit topic because … .
Jane Doe said the audited entity experienced several challenges while participating in the audit topic.
The challenges she listed are:
(1) … ;
(2) … ; and
(3) … .
John Doe said that he is most concerned with challenge 2 because … . Jane Doe said she agrees with
Mr Doe’s assessment and added that she believes … .
Source: US GAO
235
Appendix 9: Example of a record of analysis
Title Comparison of document X to document Y
Purpose To document the comparison of the documents, to include

similarities and differences
Source Document X
Document Y
Analysis/summary:
Summary of the results of the comparison
Similarities (found in both document X and Y)
1. Documents X and Y use the same descriptive language for the audit topic.
2. Documents X and Y have appendices of templates that organisations can use to document their contributions to
the audit topic.
Differences
1. Document X has a two-page section that describes best practices that organisations should follow while
participating in the audit topic.
2. Document X has an additional appendix that has examples of how a specific organisation implemented a best
practice while it was participating in the audit topic.
Similarities
Methodology: to determine the similarities between both documents, the team conducted a
side-by-side comparison and electronically searched each document for key terminology.
Documents X and Y use … and … to describe the audit topic
See page 3, third paragraph in document X, for the description of the audit topic. … was used
in this description.
See page 10, fifth paragraph in document Y for the description of the audit topic. … was used
in this description.
Documents X and Y have appendices with the same templates
See page 28 in document X for Appendix IX. The summary paragraph before the template says
that organisations can use this template to document their contributions to the audit topic.
See the following two pages (29-30) for the template.
236
See page 35 in document Y for Appendix X. The title of the appendix is ‘Sample template for
organizations to use to document contributions to …’. See pages 35-36 for the template.
The templates in Appendix IX of document X and Appendix X of document Y are the same
template.
Differences
Methodology: to determine the differences between both documents, the team conducted a
side-by-side comparison and electronically searched each document for key terminology.
Document X has a section that describes best practices
Page 13 through 14 of document X contains a section that describes best practices that
organisations should follow while participating in the audit topic.
Document Y does not have this section. See pages 2 through 20 for the term ‘best practice’.
Source: US GAO
237
Appendix 10: Example of a data reliability assessment
Record of data reliability assessment
Complete this form for performance audits.
• Data reliability is a critical part of SAI’s work and should be discussed as early as possible in the
engagement process, preferably early in the planning phase. A similar discussion should occur as the audit
team is conducting analysis and beginning to develop the findings of the audit report when the team and
internal stakeholders need to determine whether the evidentiary data are sufficiently reliable, understand
the nature of any data limitations and discuss any additional data reliability work that needs to occur.
• This form documents the team’s determination regarding the need to conduct an assessment and, as
applicable, the data reliability plan and the steps taken to implement the plan, how the data will be used as
part of the analytic basis for the findings and conclusions, and any limitations given the intended use of the
data.
Audit title:
Preparer: Technical support provided by (if applicable):
The manager signs this form either after: (1) a determination is made that a data reliability assessment is not
needed and Section I of this form has been completed or; (2) all data reliability work is complete and Sections
I, II and III of this form have been completed.
Note: If the team subsequently determines that a data reliability assessment is needed after initially determining
it was not, the team should annotate Section I accordingly, complete Sections II and III, and add a second
signature and date below after the work is complete.
Manager’s approval
Manager Date of signing
Section I: Is a data reliability assessment needed?
Data reliability assessments should be used when any computer-processed data that the team plans to use are
expected to materially affect findings (answers to audit questions), conclusions or recommendations. The
decision of materiality involves the professional judgement of the engagement team.
While a team needs to document its determination on this form, a team does not need to conduct an assessment
of the data as provided in Sections II and III if one of the following conditions applies:
238
Section II: Continued
SOURCE FOUR: Name/description of data source
Plan for assessing the data reliability of this data source, in accordance with Government Auditing
Standards 6.06 (describe plan or provide document references)
Data from this source are expected to be used in the final product in the following manner:
____ Sole support for findings, conclusions, or recommendations.
____ One of multiple sources of evidence to support the findings, conclusions, or recommendations.
____ Contextual or background information that is expected to materially affect the report’s findings,
conclusions, or recommendations.
Describe data elements assessed from this data source (provide description or provide document
reference(s)):
In table below, check all steps taken to determine if the data elements from this source are reliable
and include document reference for each of the steps below. Not all steps are required.
Check Data Reliability Steps Document Reference(s) or DM

steps taken link(s)
Review of related documentation
Interviews with knowledgeable audited entity personnel
Electronic or manual data testing for missing data,

outliers, obvious errors (could include comparison to
published data and logic tests)
Review of related internal controls
Traced selection or random sample to or from source

documents
Other (explain)
Note: If more than four sources are used, block copy this last section as needed and provide the required
information for the additional sources.
239
Section III: How can you use the data?
Summarise the findings on the reliability of data from each data source. Include information
on data limitations, if any, and how those limitations will affect how the data will be used in
the product (for example, the effect on the findings, conclusions, or recommendations).
Considering the findings on the reliability of data from all sources assessed (check one):
____ All data elements we assessed are sufficiently reliable for this engagement (the
limitations, if any, are described above).
____ Some data elements we assessed are sufficiently reliable, and the limitations, if any, are described
above. Those data elements that are not sufficiently reliable are excluded from this engagement.
____ No data elements are sufficiently reliable for this engagement, and they are excluded
from this engagement.
____ Undetermined reliability, limitations, and their effect are described above.
____ Other (for example, primary objective was to assess the reliability of a system or part of a system)
(explain).
Note: After Sections II and III have been completed following a determination that a data reliability
assessment was required, the manager reviews the form and approves the data reliability assessment
by signing on page 1.
240
Appendix 11: Sample data reliability questions for the audited entities
1.When was the data system created, and what is its purpose?
2.How does the data owner use the data?
3.Who are the data system’s primary users?
4.How do users access the system?
5.Who has access to enter or update the data?
6.Are there different ‘levels’ of access to the data?
7.What, if any, training is provided to system users?
8.Is training made available to all users?
9.Have there been any changes to the data system (for example, major system upgrades, changes to new
vendors) that would affect the consistency of data during the time period requested?
10. How and where are data collected (for example, manual data entry, form completed by agency
representative, entry by entities outside the data owner)?
11. Who is responsible for data entry?
12. How current are the data?
13. How frequently are data entered?
14. What instructions does the data owner provide for data entry, particularly for data fields that are open-
ended or otherwise subject to variation in user input?
15. What is known about the consistency of data entry across staff, offices or other units?
16. If data are produced by aggregating across units (for example, states, organisations), are there differences
in how the units collect or calculate the data that might result in inconsistencies within the data once
aggregated?
17. Are data entries subject to change, either because of quality reviews or other procedures? What unit of
analysis does each record in the data represent (for example, an individual, event, household)?
18. What is the structure of the data system?
19. Are data maintained in a ‘flat file’, or is the data system relational/hierarchical?
20. If the data are relational, what unique identifier(s) are used to link the tables?
21. Are any data (either records or fields) in this dataset fed in from other data systems?
22. If any of these data are fed in from another data system, what quality control features are in place to ensure
data are read inaccurately and completely?
23. What procedures ensure the data system consistently captures all data occurrences (records,
observations) and all data elements?
24. What procedures are in place to prevent duplicate records being created in the data?
25. Does the system have any edit checks or controls to help ensure the data are entered accurately?
26. Are there electronic safeguards, such as error messages for out-of-range entries or inconsistent entries?
27. Does someone review all, or a sample of, data entries to ensure key fields are accurate and non-
duplicative? If reviews take place, how frequently do they occur?
28. What process, if any, is used to track and oversee changes made to the data?
29. Does the data system maintain a history of the changes made to the data, or is historical information
overwritten when new data are entered? If data are contained in a spreadsheet: what procedures are in
place to ensure data are not inadvertently changed or deleted, and are any formulas in the spreadsheet
reviewed for accuracy?
30. What are the procedures for follow-up if errors are found, and who is responsible for correcting them?
241
31. To the extent you have identified errors in relevant data fields, what were the reasons for the errors and
have these issues been addressed?
32. Do systematic reviews or exception reports examine accuracy and present error rates? How frequently?
33. If studies or evaluations of the system have been conducted, what were the results, and how did you
address any issues?
34. If applicable, do external users of the data or individuals who are the subject of data records have the
opportunity to review and provide feedback on data accuracy?
35. Are any new variables created by recoding existing variables or calculated based on values for existing
variables (for example, calculation of number of days between recorded dates or creation of a variable
based on age ranges)?
36. Does data system documentation explain how new variables are created or calculated?
37. What modifications, if any, are made to data values in order to protect confidentiality or for other
purposes?
38. Do any variables use categorisations developed by another organisation (for example, categories of
industry type or race)?
39. Have there been changes to any procedures – including how a data element is defined, entered or
maintained – over the period for which data are requested (for example, changes to populations or
geographic areas, variable definitions, variable values or categories, data entry instructions, available drop-
down values)?
40. If there have been changes to procedures within the time for which data are requested, what steps have
been taken to ensure the accuracy and consistency of the data?
41. What is your opinion of the quality of the data, specifically its completeness and accuracy? Are there any
data limitations, such as data elements, that are often incomplete or incorrect? How would those
limitations affect the intended use of the data?
42. Are there concerns about timeliness or usability of the data?
43. Are there any purposes for which the data should not be used?
44. Have any corrective actions been taken to improve the quality of the data?
Source: US GAO report, Assessing Data Reliability, 2019, GAO-20-283
242
Appendix 12: Sample data collection instrument
This data collection instrument (DCI) is an example for an audit team reviewing a
government organisation’s agreements with implementing partners/participants.
Data Collection Instrument (DCI)

DCI first completed by:
Date first completed:
DCI reviewed by:
Date of review:
A) Basic Information
1. Document name
2. Document date
3. Originating source
B) Details
1. Implementing partner/participant
2. Year of agreement
3. Single or multi-year agreement? Single Multi-year Cannot determine
4. (If a multi-year agreement) How many years did the agreement cover
Note: When creating a DCI, consider data field design, formatting and measurement, to include:
• How, if at all, will the team aggregate the information from each data field in the DCI?
• Will the team use one DCI per case or one DCI for all cases?
• What staffing and data collection procedures are needed (for example, execution of onsite verification
and/or review of the data entry; allow space on the DCI for sign-off or initialling)?
• What will be the likely sequence of the data fields on the DCI (which information will be collected first,
second, etc.)?
• Will the DCI use open-ended data fields to capture additional or unexpected information, such as
document titles, additional observations or onsite review of paper documents that the team cannot copy
or annotate?
• How will the format for each data field match the type of desired information: checkboxes, multiple-
choice options (for example, Yes/No), fill-in-the-blank text boxes?
• If a paper form is used for initial data collection in the field, how will the data be transferred to an
electronic file? In such cases, consider how to match the layout of the paper and electronic forms.
Source: US GAO
243
Appendix 13: Sample template for documenting direct observations
Title Observation of customs inspections conducted by officials of audited

entity ...
Purpose To document observations during customs inspections
Place observed Airport …
Activity observed Number of customs inspections
Date of observation Type date here
Participants Audited entity X
Official 1
Official 2
Supreme Audit Institution
Auditor A
Auditor B
Observations/remarks:
We observed a total of … customs inspections throughout this time frame. Details about these inspections can
be found below.
(1) Inspection by Official 1 from 0800-0830:
• Official 1 began by opening the handbag of the individual subject to the inspection. She proceeded to
empty the entire handbag’s contents onto the table and sort through the items. As she sorted through
the items, she systematically consulted a checklist of materials that were not supposed to be brought
into the country. She did not find any prohibited items in the subject’s handbag.
• She then proceeded to open the subject’s suitcase. She sorted through the items found in the suitcase
by moving items found on top of others to the side. As she sorted through the items, she also consulted
the same checklist that she had used for the handbag. During this search, she found one item that was
listed on the checklist. She proceeded to place it to the side and returned to her search of the luggage.
After she went through the rest of the suitcase, she asked the subject about the prohibited item that
she had placed to the side. She used another separate checklist of questions to query the subject about
the prohibited item.
Source: US GAO
244
Appendix 14: How to conduct a survey
Identify the survey population

You need to identify the population you will survey. In doing so, you need to ensure that the
individuals or organisations you identify are the best sources of the information you are
hoping to obtain.
You will also need to determine how many individuals or organisations you will survey. For
some audits, the target population may be small (for example, an organisation with 100
employees), and thus you can reasonably survey the entire population.
The target population may be very large (for example, one million citizens who receive
support from a government entity). In the case of a large target population, you may be able
to survey only a sample of the population. In such cases, you need to ensure you have the
appropriate sample to use the information obtained for your desired purposes. This can be a
complicated process, so it is recommended that you seek the advice of a survey expert.
Developing the survey questionnaire

Developing the right survey questions is critical to obtaining quality information that you can
use as evidence. Here are some steps to help you develop a questionnaire:
1. Determine which portions of your audit question(s) will be addressed using the survey.
It is important to design the survey to directly assist you in answering your audit questions.
If a survey cannot help you do this, consider another method.
2. Break down those portions of the audit question(s) to a set of topic areas and then
develop questions that address topic areas with increasing levels of specificity (see Figure
14.1). Questions can be open-ended or closed-ended, depending on your need for
information. Regardless, it is recommended that the questions:
• be written so that respondents can easily and consistently interpret them – that is, short
and simple;
• be written so that respondents have access to the information needed to answer them;
• not be overly burdensome for the respondent to answer; and
• not be written to bias the respondent’s answers.
245
Figure 14.1: An example of developing survey questions from an audit question
Audit question:
How effective is the Department of Veterans Affairs outreach to
veterans and service members applying for education benefits,
especially for those individuals with disabilities?
Increased specificity (focus on website):

How well or poorly did the Department of Veterans Affairs
website describe the range of education benefits available for
veterans and service members?
More specific (focus on one aspect of the website):

How helpful, if at all, was the Frequently Asked Questions section
of the Department of Veterans Affairs website in terms of
answering your specific questions on veterans education
benefits?
Possible message based on survey questions:

One of the Department of Veterans Affairs’ main outreach
channels is their website. Most service members and veterans
surveyed indicated that the website was an effective outreach
mechanism. However, the website was seen to be lacking
fundamental educational benefit information veterans expect to
be readily available.
Source: US GAO
It is important to pretest, evaluate and refine the survey questions. It is recommended that
you:
• Pretest your draft survey questionnaire with members of the targeted survey population
and obtain feedback from those individuals about whether they understood the questions.
• Evaluate the responses to the pretested survey to determine whether the questions you
are asking will elicit the data you need.
• Consider how the survey responses might allow you to answer the audit questions in the
report.
• Refine your questions based on the pretest(s) and evaluation until you are confident that
you are asking the right questions of the survey population.
Select a method for administering the survey
There are multiple methods you can use to administer a survey, including face-to-face or
telephone interviews, web-based surveys, paper surveys via mail, electronic surveys via email,
or in-person self-administered paper surveys.
The population size, your staff resources and how you will contact the survey respondents
are all important factors. Here are some questions to consider:
• Does the population have access to internet, telephone and mail service?
246
• Do you have accurate contact information for the target population for your chosen
method of communication (for example, phone numbers, email addresses, mailing
addresses)?
• Do members of the population have any challenges with reading, vision, hearing or
mobility that could affect their ability to take the survey via different methods?
• How large is the target population, and do you have sufficient staff resources to consider
an interviewer-administered option?
The method you choose will affect the response rate to your survey if the target population
cannot easily respond to the survey or if you do not have the staff resources to administer it
as planned.
Documenting the survey results and methodology
You will need to carefully document how you conducted the survey, the survey responses and
any analysis performed on the survey results. This is important because you will need to
provide support for all statements in the final report based on evidence obtained from the
survey. Additionally, you will need to provide information in the audit report about the survey
methodology, quality of the data obtained from the survey, and the strengths and
weaknesses in the survey so that those who read your report understand how to interpret
the survey results you provide.
Conducting an effective survey will require more guidance than this handbook can provide.
Remember to seek out assistance from an expert before attempting to conduct a survey.
247
Appendix 15: Content analysis
Content analysis is a qualitative method for structuring and analysing complex qualitative
data and turning it into quantitative data. It is sometimes described as a process of data
reduction. The goal is to systematically sort, focus and simplify data into a limited number of
themes or content categories that can be summarised. Because it can be time-intensive, it
may not be as commonly used by some SAIs as some of the other qualitative methods
referenced in this handbook, but it can be useful in certain situations.
The qualitative data used as a starting point for a content analysis could include the audit
entities’ policy documents, interview transcripts, newspaper articles, focus group transcripts,
claim files, or reports. For example, you could use it to categorize and quantify the responses
provided by interviews or determine the frequency with which different types of events were
reported in claims files. Content analysis can also be a useful method if you have a large set
of raw data that you need to turn into useable evidence, such as survey responses. The
example in Figure 15.1 is adapted from a content analysis of survey responses conducted
during a problem-oriented SAI performance audit.
Figure 15.1: Content analysis used in a performance audit of actions taken to confront
domestic violence against women
Auditors collected survey responses from 340 people who support women victims of violence, such
as police officers, psychologists and social workers. The final question in the survey was, “In your
opinion, what should be done to improve the services to women victims of violence and to
decrease this type of violence in our country?”
The audit team performed a content analysis of the survey responses and then categorised the
responses. The six most popular categories are shown below.
Increase qualifications of staff
Provide more staff
Work with offenders

Increase prevention education
Increase law enforcement

Increase awareness of laws
Source: Adapted from the Performance audit report: Ações de enfrentamento à violência doméstica e familiar contra as mulheres. (Actions to face
domestic and familiar violence against women), 2012. SAI Brazil
There are a number of potential benefits of conducting content analysis, including that the
categories or themes that result from the content analysis can be summarised and reported
in ways that are easily understood by readers.
Content analysis that produces reliable data can be time and labour intensive, depending on
the complexity of the analysis. It is important to conduct content analyses systematically, so
248
talk to a methodologist or other internal stakeholder with subject matter expertise, or consult
academic literature, for additional guidance, as needed.
249
Appendix 16: Sample template for documenting a summary
Summary of perspectives on the sufficiency of training for

Title
customs inspectors
To provide a summary for evidentiary purposes

Purpose
During this audit, we conducted multiple interviews to gather
Work performed
information on the sufficiency of training for airport customs
inspectors. Specifically, we interviewed:
• Administrators of the training program at the Customs
Inspections Academy.
• Instructors at the Customs Inspections Academy.
• Officials who developed the training curriculum for
customs inspectors.
• Customs inspectors at three airports.
• Supervisors of customs inspectors at three airports.
We asked each of these groups for their perspectives on 1) length

of the initial training; 2) content of the initial training; and 3) on-
the-job training after instructors begin working. This summary
compiles responses of the officials relative to each of these audit
topics.
Summary of responses
(In the table below, the audit team would compile the responses of officials on these audit topics.
See examples below.)
Length of the initial training
Administrators The administrators of the training program at the Customs Inspections

Training Academy said that the length of training program is long
enough to provide inspectors with a base level of training proficiency.
The administrators said that the Academy does not have enough
funding to extend the training. (See document xxxx, pg. 3)
Instructors The instructors at the Customs Inspections Training Academy said that
the initial training is not long enough. At minimum, the instructors said
that they would need another two weeks to allow for time for more
hands-on exercises and time for review. The training calendar now is
too rushed, and some trainees fall behind. (See document XXXX, pg. 2)
Curriculum developers (add summary of responses)

Custom inspectors (add summary of responses)
Customs supervisors (add summary of responses)

Content of the initial training

Training Academy said that the content of the initial training is
250
sufficient, but there may be areas where it can be improved. In fact, the
Academy is beginning a review of the training curriculum in March
2021. It has a goal of reviewing the curriculum and making any needed
revisions every two years, but this does not always occur. The last
review and update was completed in August 2017. (See document
XXXX, pg. 7)
Instructors (add summary of responses)


On-the-job training

Training Academy said that the Academy has not established formal
guidance on on-the-job training. They rely upon the supervisors to
determine what the inspectors need and provide it. (See document
xxxx, pg. 9)
Instructors (add summary of responses)


251
Appendix 17: Example of a regression analysis
The simplest form of regression analysis is often referred to as correlation analysis. This type
of analysis may be useful to you if you are trying to determine how two different variables
are related to one another – that is, the degree to which changes in one are associated with
changes in the other.
There are three general steps involved in a correlation analysis:
1. Development of a scatter diagram, which plots values of the dependent variable ‘Y’ and
independent variable ‘X’ on vertical and horizontal axis, respectively. The dependent
variable is the variable that is being predicted or estimated, and the independent variable
is the variable that provides the basis for estimation.
2. Calculating the correlation coefficient (r), which measures the correlation between the
variables. The closer the correlation coefficient is to 1 or -1, the more the two variables are
correlated. In a perfect positive or negative correlation, all the dots in the scatter plot
would form a straight line.
3. Calculating the coefficient of determination (r2), which measures the extent to which the
variation in the dependent variable can be explained by variations in the independent
variable.
The following example was adapted from an audit conducted by the Supreme Audit
Institution (SAI) in Bhutan. It will provide you with a simple application of this type of analysis
to illustrate its potential usage.
Example: The SAI conducted an audit that examined the relationship between the number of
paediatricians and child mortality, based on the goal of the health sector to reduce infant
mortality. Here are the data the audit team used:
Year 2003 2004 2005 2006 2007 2008

Number of paediatricians 10 15 20 20 25 30
Child mortality 300 310 280 251 300 200
Source: SAI Bhutan
The data set produced the scatter diagram in Figure 17.1.
252
Figure 17.1: Scatter diagram
Child mortality
350
300
250
Linear (child mortality)
200
150
100
50 Child mortality
0 Number of
0 10 20 30 40 paediatricians
We can see from this linear trend line that there is some correlation between child mortality
and the number of paediatricians. Still, we want to understand how closely the two variables
are correlated. To do this, we need to calculate the correlation coefficient, or ‘r’. It can be
done using the ‘CORREL’ function on a spreadsheet program.
The ‘r’ value is -0.712. This means there is a strong negative correlation between the number
of paediatricians and child mortality – that is, as the number of paediatricians increases, child
mortality decreases.
Just because there is a strong correlation, though, does not mean there is causality. We need
to also calculate the coefficient of determination, or ‘r2’, to determine how much of the
variation in child mortality can be explained by the number of paediatricians.
In this case, r2=0.507, or 50.7%. So, in our example, 50.7% of the variation in child mortality is
explained by the number of paediatricians available, and 49.3% of the variation is due to other
factors.
As you can see from this example, there are many factors that influence changes in a
dependent variable like child mortality. More complex modelling and regression techniques
that address or control other variables would be necessary for the audit team to fully
understand the variables affecting child mortality.
253
Appendix 18: Sample GAO highlights page
OCTOBER 2019
INFORMATION TECHNOLOGY
Highlights Agencies Need to Fully Implement Key Workforce
Planning Activities
Highlights of GAO-20-129, a report to
congressional requesters.
Why GAO Did This Study What GAO Found
The federal government annually Federal agencies varied widely in their efforts to implement key information
spends over $90 billion on IT. Despite technology (IT) workforce planning activities that are critical to ensuring that
this large investment, projects too agencies have the staff they need to support their missions. Specifically, at least
frequently fail or incur cost overruns 23 of the 24 agencies GAO reviewed partially implemented, substantially
and schedule slippages while implemented, or fully implemented three activities, including assessing gaps in
contributing little to mission-related competencies and staffing. However, most agencies minimally implemented or did
outcomes. Effectively implementing not implement five other workforce planning activities (see figure).
workforce planning activities can
facilitate the success of major Agencies Overall Implementation of the Key Information Technology (IT) Workforce Planning
acquisitions. Activities
Set the strategic direction for IT workforce planning
GAO was asked to conduct a Establish and maintain a workforce planning process
government-wide review of IT 1 1 2 12 8
workforce planning. The objective was
Develop competency and staffing requirements
to determine the extent to which federal
12 4 8
agencies effectively implemented IT
workforce planning practices. To do so,
GAO compared IT workforce policies Analyze the IT workforce to identify skill gaps
and related documentation from each Assess competency and staffing needs regularly
of the 24 Chief Financial Officers Act of 3 20 1
1990 agencies to activities from an IT Assess gaps in competencies and staffing 1
workforce planning framework GAO 2 9 12
issued. GAO rated each agency as
having fully, substantially, partially,
Develop strategies and implement activities to address IT skill gaps
minimally, or not implemented for each Develop strategies and plan to address gaps in competencies and staffing
activity. GAO supplemented its reviews 6 13
4 1
of agency documentation by
Implement activities that address gaps
interviewing agency officials.
2 7 15
What GAO Recommends Monitor and report progress in addressing IT skill gaps
GAO is making recommendations to 18 Monitor the agency’s progress in addressing gaps
of the 24 federal agencies to fully 3 5 16
implement the eight key IT workforce Report to agency leadership on progress in addressing gaps
planning activities. Of the 18 agencies, 3 3 18
13 agreed with the recommendations,
one partially agreed, three neither 0 6 12 18 24
agreed nor disagreed, and one Number of agencies implementing the activity
disagreed with the findings and
Fully implemented Substantially implemented Partially implemented
provided evidence which led to a
modification to its recommendation, as Minimally implemented Not implemented
discussed in this report. For all of the
remaining recommendations, GAO
continues to believe that they are all
Source: GAO analysis of agency information technology workforce planning policies and documentation. | GAO-20-129
warranted.
Agencies provided various reasons for their limited progress in implementing workforce planning
View GAO-20-129. For more information, activities, including competing priorities (six agencies) and limited resources (three agencies).
contact Carol C. Harris at (202) 512-4456 or Until agencies make it a priority to fully implement all key IT workforce planning activities, they will
[email protected] likely have difficulty anticipating and responding to changing staffing needs and controlling human
capital risks when developing, implementing, and operating critical IT systems.
_______________________________________ United States Government Accountability Office
254
Appendix 19: Sample European Court of Auditors executive summary
Executive summary
I The Common Agricultural Policy has a long history of using satellite or aerial images for checking
area-based aid, which nowadays accounts for almost 80% of the EU funding provided to agriculture
and rural development. While these images usually have a very high spatial resolution, before 2017,
they were not available with sufficient frequency to allow verification of activities taking place on
agricultural land throughout the year (e.g. harvesting).
II Since March 2017, the EU-owned Copernicus Sentinel satellites 1 and 2 have been providing
frequent, freely available, high-resolution images, with the potential to be a game-changer in Earth
observation technology for monitoring agricultural activities. Since the images are taken frequently,
automated processing of time series data throughout the growing season makes it possible to identify,
without human intervention, crops and monitor certain agricultural practices on individual parcels
(such as tillage, mowing). Since 2018, paying agencies can use Copernicus Sentinel data in place of
traditional checks based on field inspections.
III According to the commission and CAP stakeholders, Copernicus Sentinel data and other
technologies for monitoring area aid have significant potential benefits for farmers, administrations
and the environment. Our audit examined whether the commission effectively encouraged
widespread use of these new technologies and whether Member States had taken adequate action to
deploy them. We looked at the Copernicus Sentinel satellite data, images taken by drones, and
geotagged images. An assessment of the progress made in the use of new imaging technologies is
especially relevant now, as the results of our audit could be applied in the post-2020 CAP.
IV We found that both the commission and some Member States have taken action to unlock the
potential benefits of the new technologies. The commission promoted new technologies through
many conferences and workshops and provided bilateral support to many paying agencies. 15 out of
66 paying agencies used the Copernicus Sentinel data in 2019 to check aid applications for some
schemes and some groups of beneficiaries (‘checks by monitoring‘). Our audit revealed that many
paying agencies consider obstacles to wider use of the new technologies.
V Although the commission has attempted to remove or mitigate some of these obstacles, paying
agencies expect further guidance from the commission to make the right decisions and reduce the risk
of future financial corrections.
VI Moving to checks by monitoring requires significant changes to IT systems, specific resources and
expertise. The commission has taken initiatives to facilitate access to Sentinel data and digital cloud
processing services, but the take-up by paying agencies for operational purposes is still low.
VII With regard to rural development schemes and cross-compliance, we observed limited use of new
technologies for both compliance and performance monitoring of climate and environmental
requirements. We also conclude that the proposed set of post-2020 CAP performance indicators is
largely not designed for direct monitoring with Sentinel data.
VIII We recommend that the commission provide incentives to Member States to use checks by
monitoring in the post-2020 CAP as a key control system. We further recommend that the commission
make better use of new technologies for monitoring environmental and climate requirements.
255
Appendix 20: Description of an audit methodology in a performance audit report
To examine the characteristics of FAA-certificated mechanics and repairmen, we analysed

cumulative FAA data as of December 2018 for demographic characteristics such as age and
sex.4 To examine the employment characteristics of aviation maintenance workers—such as
wages and unemployment—we analysed Bureau of Labor Statistics (BLS) Current Population
Survey data for selected labor market indicators from 2013 through 2018. We reviewed all 50
states’ most recent Workforce Innovation and Opportunity Act plans.
To describe stakeholder support and assess stakeholder coordination on efforts to develop
this workforce, we interviewed agency officials from FAA and the Departments of Labor
(DOL), Education (Education), Defense (DOD), and Veterans Affairs (VA) about related data,
programmes, and funding for this workforce. We selected these agencies based on a prior
report in which we identified them as relevant to the aviation workforce.
To describe examples of stakeholder coordination, we also conducted semi-structured
interviews with a non-generalizable sample of six stakeholders, including employers, Aviation
Maintenance Technician (AMT) Schools, unions, industry associations, and workforce training
organizations selected based on stakeholder recommendations, among other factors, and
conducted two site visits. We visited an AMT School that serves the District of Columbia area
and an aviation repair station, a major commercial airline, and a state workforce organization
in Georgia. We selected these stakeholders and conducted these site visits to obtain a range
of perspectives based on factors such as type of work performed and geographic location. In
addition, we reviewed relevant agency documents, such as FAA’s 2019-2022 strategic plan
and its Aviation Workforce Steering Committee charter.
To describe what progress FAA has made on updating training curriculum requirements for
AMT Schools and certification testing standards for mechanics, we reviewed relevant federal
laws, regulations, and FAA documents and interviewed FAA officials.
We conducted this performance audit from January 2019 to February 2020 in accordance
with generally accepted government auditing standards. Those standards require that we
plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We believe that the
evidence obtained provides a reasonable basis for our findings and conclusions based on our
audit objectives.
Source: US GAO report, AVIATION MAINTENANCE: Additional Coordination and Data Could Advance FAA Efforts to
Promote a Robust, Diverse Workforce, 2020
256
Appendix 21: Illustration of an action plan and a follow-up desk review template14
Action plan
Title of performance audit: Performance audit on elimination of intimate partner violence

against women (EIPVAW)
Name of audited entity: Secretary of Policies for Women
Date: 20/11/2017
Actions planned by Responsible party Deadline for Expected benefits of
Recommendation the audited entity within the audited implementing implementation
entity recommendation (quantified if possible)
Conduct Plan and deliver a Ms Shirley Smith. September 2018. Decrease of intimate
awareness-raising campaign through partner violence
campaigns about social media. against women.
the importance of
EIPVAW. Plan and deliver a Mr Juan Perez. December 2018.
campaign for TV.
Intensify campaigns Plan and deliver a Ms Shirley Smith. September 2018. Help in changing
about EIPAW aimed campaign through men’s mentality
at males. social media. about violence
Mr Juan Perez. December 2018. against women.
Plan and deliver a
campaign for TV.
Decrease of intimate
Mr Abdalla Farid. December 2018. partner violence
Plan and deliver a
campaign during sport
against women.
matches.
Coordinate with the Contact stakeholders in

Ms Caterina Piazza. March 2018. Decrease of intimate
the Ministry of
Ministry of partner violence
Education.
Education to against women.
include gender Plan the curriculum Ms Caterina Piazza. September 2018.
themes in the changes with the
school curriculum, Ministry of Education
especially issues stakeholders.
related to domestic
violence. Deliver training for Mr Joshua Pereira. December 2019.
teachers about the
changes in the
curriculum.
Coordinate with the Contact stakeholders in Mr Juan Perez. March 2018. Improvement in
Ministry of Justice to the Ministry of Justice. service delivery by
intensify awareness the police officers to
raising and training Plan and deliver a Ms Shirley Smith. September 2018. victims of domestic
of police officers campaign through violence.
who attend the internal social media.
victims.
Deliver training for Ms Chimamanda Nye. December 2019.
police officers.
14 This is an illustration with sample recommendations. It is not intended to be exhaustive. An actual performance audit
will likely contain additional recommendations.
257
Follow-up desk review template
Illustration using a performance audit on elimination of intimate partner violence against

women (EIPVAW)
Action taken by the Status/progress of Reasons for non-
Impact
Recommendation audited entity (as actions completion of
per the action plan) action by the
audited entity
Conduct Plan and deliver a Fully implemented. The cost is too high. The Secretary of
awareness-raising campaign through The Secretary has Policies for Women
campaigns about social media. Not implemented. no budget for that. contracted a
the importance of consultant to
EIPVAW. Plan and deliver a measure the impact
campaign for TV. of the campaigns.
The study will be
concluded in July
2021. The impact has
to be verified by the
SAI during next follow-
up.
Intensify campaigns Plan and deliver a Fully implemented. The cost is too high. The Secretary
about EIPAW aimed campaign through The Secretary has conducted surveys
at males. social media. no budget for that. after several football,
cricket and
Plan and deliver a Not implemented. basketball matches
campaign for TV. where the campaign
took place. The
Plan and deliver a Fully implemented. preliminary analysis
campaign during shows that the
sport matches. campaign was
successful in raising
awareness about
EIPAW among the
respondents.
Coordinate with the Contact stakeholders Fully implemented. The group assigned Since the changes
Ministry of in the Ministry of for the task is in the curriculum
Education to Education. developing the were not made yet,
include gender Implemented in some changes, but there it is too early to
themes in the Plan the curriculum respects. are delays due to evaluate the
changes with the
school curriculum, other urgent impact of the
Ministry of Education
especially issues assignments. actions.
stakeholders.
related to domestic Not implemented.
violence. Deliver training for Not due. To be
teachers. verified in next
follow-up.
Coordinate with the Contact stakeholders in Implemented. The training is The Secretary
Ministry of Justice to the Ministry of Justice. completed only for conducted a study to
intensify awareness police officers compare attendance
raising and training Plan and deliver Implemented. working in major by police officers
campaign through trained and not trained
of police officers cities of the country.
internal social media. and concluded that
who attend the The others are yet
the training, so far, was
victims. Implemented in some to be trained. effective.
Deliver training for
police officers. respects.
258

Performance Audit: ISSAI Implementation Handbook

Uploaded by

Copyright:

Available Formats

Performance Audit: ISSAI Implementation Handbook

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance Audit: ISSAI Implementation Handbook

Uploaded by

Copyright:

Available Formats

Performance

Figure 1: Relationship between the 3Es .......................................................................................................................... 18

3i Audit Programme ISSAI Implementation Initiative Programme

Quality Assurance Protocol: Version 2.01

Updates to this GPG

1 The protocol is available at http://www.idi.no/en/idi-library/global-public-goods

Quality Assurance Review Process

Results of the Quality Assurance Review

Mr. Einar Gørrissen

What is the purpose of the handbook?

2 International Standards of Supreme Audit Institutions.

How was the handbook developed?

Contents of the handbook

Before beginning a performance audit, it is important to understand what a performance audit

This chapter will answer the following questions:

What is performance auditing?

Source: ISSAI 3000/17

• Activities, for example, procurement policies across government.

What are the objectives of a performance audit?

What are the 3Es – economy, efficiency and effectiveness?

Efficiency: Making the most of available resources

7 Adapted from ECA Performance Audit Manual, 2017.

Effectiveness: Achieving the stipulated aims or objectives

Source: ISSAI 300/11

8 Adapted from ECA Performance Audit Manual, 2017.

9 Adapted from ECA Performance Audit Manual, 2017.

What is the relationship among the 3Es?

Figure 1: Relationship among the 3Es

Source: Adapted from European Court of Auditors

How does performance auditing promote accountability and transparency?

What value do performance audits bring?

What value do performance audits add?

Generally, performance audit offers benefits such as identifying:

What types of reports result from performance audits?

Determining whether a Health Ministry Performance audit report on

Determining how well public bodies are Green public procurement – is

Assessing whether government support Has European Regional Development

Examining the extent to which gambling Gambling regulation: problem

Assessing whether the responsible The management of water distribution

Assessing whether the responsible Sustainable management of fish

Assessing whether the government,

Determine the extent to which the GASTPE. SAI Philippines, 2018.

Determining whether the services Prevention services on pregnancy of

Source: ISSAI 3000/25

Source: ISSAI 3000/29

Source: ISSAI 100/35

Tone at the top of the Supreme Audit

Audit work and reports

Audit work should adhere to SAIs should only carry out

Source: ISSAI 140

Adhering to independence and ethical requirements is a prerequisite to conducting performance

Independence means being free from circumstances or influences that

Source: IDI/PAS Development Team

Source: GUID 3910/14

• not accepting gifts, bribes or

Source: GUID 3910/19

Ethics are the moral principles of an individual that include integrity,

SAIs should have policies and procedures in place that:

Source: IDI/PAS Development Team