Ten Essential Audit Questions

Revealing a system's effectiveness and an organization's overall performance

by Craig Cochran

All experienced auditors accumulate favorite audit questions, and I'm no exception. I have
a short, punchy list of queries I invariably ask while evaluating a management system.
Favorites aside, though, what are truly the most important audit questions? What
questions will reveal a system's effectiveness and an organization's overall performance?
I compiled a list of the top 10.

1. How do you contribute to achieving your organization's objectives?

ISO 9001:2000 requires that organizations establish measurable objectives at relevant

functions and levels. Perhaps an even more significant requirement is that personnel
understand how they contribute to these objectives. This requirement doesn't apply only
to some employees; it applies to everyone. All personnel must be able to communicate, in
their own words, how they help move objectives in the right direction. It's conceivable
that not all objectives apply to everyone in the company, and in those cases auditors
would only expect that personnel understand the objectives that apply to them.

This question directly reflects on an organization's ability to communicate what matters

most to its success. Truly comprehending objectives means that people understand
specifically what they can do to improve the organization. They appreciate the
significance of their roles and are prepared to carry them out. This knowledge creates
strategic focus throughout the organization. Instead of having a limited view of activities
and tasks, personnel begin to understand how their jobs link to the organization's larger
mission.

Closely related questions include:

 How are objectives determined?

 How are employees trained on objectives?
 How is progress against objectives communicated to the organization?
 What processes and/or tools are in place to help achieve objectives?
 Is there evidence of progress?

2. What happens if your product, materials or supplies are nonconforming?

This question reflects on the organization's ability to deal with product problems in a
systematic way. Controlling nonconforming products is a basic discipline and one that
smart auditors always probe. The answer to this question can be compared to the
documented procedure and, more important, to the auditor's observations.

Few other processes require as rigid adherence to procedures as controlling

nonconforming products. Simply put, there's no room for deviation. Problems relating to
controlling nonconforming products almost always pose significant risks to the
organization-in added costs, wasted time, aggravated employees, angry customers and
loss of competitive position.

During an audit, find some examples of nonconforming products--if any exist--and follow-
up with these questions:

 How are nonconforming products identified?

 Where are they located?
 What are the responsibilities and authorities related to dealing with
nonconforming products?
 How do dispositions get determined and implemented?
 What are the records of nonconforming products and actions taken on them?
 What are the trends in nonconforming products?
 How is the procedure linked to the corrective action process?

It's worth mentioning that controlling nonconforming products applies to services just as
much as it does to tangible goods. Reports, data, test results and intellectual property, to
name just a few service outputs, can all be potentially nonconforming, in which case all
the disciplines of this process apply.

3. How do you access product requirements?

Everybody has a product of some sort. It might go to an external customer or simply to

the next process inside the organization. In all cases, though, personnel must understand
the product requirements. ISO 9001:2000 specifically requires that organizations identify
product requirements in four ways:

 As stated by the customer

 Unstated by the customer but necessary for intended use
 As statutory and legal regulations related to the product
 As any additional requirements determined by the organization
 The standard additionally requires that information describing the product be
available (i.e., documented). Asking how personnel access product requirements
is an important audit question because when requirements aren't accessible, big
problems often result. Employees don't need to know product requirements by
heart, but they should certainly be able to find the current versions of
requirements and describe how they carry them out.

Specific points of inquiry related to product requirements include:

Are product requirements complete?

How does the organization ensure that correct versions are available?

How are requirements reviewed prior to acceptance?

How do you ensure that product meets the stated requirements?

What happens when changes are made to product requirements?

4. How are problems prevented?

Problem correction is relatively simple: Define the problem, identify the cause and take
action to remove it. Problem prevention, on the other hand, is more complex. Many
people would also argue that it's more important. Preventive action is specifically
required by ISO 9001:2000, and it provides one of the most valuable links to continual
improvement.

The most obvious way to generate preventive action is by analyzing data. Data analysis is
a primary job of top management, but it can happen at other levels of the organization as
well. When an organization openly shares data and encourages its analysis on a broad
scale, then preventive action becomes easy. Employee creativity and innovation can also
be a valuable starting point for preventive action. Savvy organizations look for ways to
solicit improvement ideas from their employees and provide feedback on the viability of
the ideas. Another source of preventive action is feedback from customers. Often,
customers will provide ideas for improving the product in subtle yet significant ways.

Additional points of inquiry related to preventive action include:

 How do data trends get analyzed?

 How do employees communicate their improvement ideas?
 How do preventive actions get recorded?
 Are statistical techniques used?
 How are customer perceptions captured on a proactive basis?

5. How do you use data on customer perceptions?

This question probably won't apply to all personnel. It's especially relevant to top
management and employees responsible for gauging customer perceptions. The question
is significant because most organizations manage fairly well to capture perceptions but
usually fall short of actually doing something with the information.

ISO 9001:2000 specifically requires that organizations define methods for obtaining and
using customer satisfaction data. This is another reason for relying on simple methods for
capturing customer perceptions. The more complex and resource-intensive your customer
satisfaction methods are, the less likely you'll take action on what you learn. It's a curious
paradox. Many organizations run out of gas before they get to the action phase, and the
valuable opportunities afforded by customer feedback are ignored as other problems
arise.

Here are some related audit questions:

 How are data on customer satisfaction analyzed?

 How are opportunities identified and prioritized?
 What's the connection to the corrective and preventive action systems?
 What are the organization's long-term trends in customer satisfaction?
  How are resources for customer satisfaction identified and provided?
 What connections exist between customer satisfaction and the organization's
objectives?

6. How are customer complaints handled?

Despite everyone's best efforts, customers will occasionally complain. Customer

complaints represent both a huge risk and valuable opportunity to the organization-it all
depends on how they're handled. This question is especially relevant to salespeople,
customer service representatives, technical personnel and top management. The auditor
is looking for proof of a systematic approach to dealing with complaints. This will typically
include defined responsibilities for logging and tracking complaints, clear problem
statements with all relevant facts included, determination of problem causes and actions
that address the causes. Specific examples of complaints must be sampled, of course.
The link between the complaint process and corrective action also requires special
scrutiny.

Here are some related questions:

 What's the largest complaint category?

 What's being done about it?
 Has the amount of complaints changed over time?
 How are personnel trained in their roles in preventing complaints?
 How are customers made aware of actions on their complaints?
 What tools are used to identify the causes of complaints?

7. How does top management review the organization's performance?

One of top management's most important responsibilities is reviewing the organization's

performance. I'm not talking about employee performances but the organization's
success as a whole. Is your organization becoming more efficient, more competitive,
better at serving customers? Or is it moving in the opposite direction? Top management
should regularly analyze data and trends that provide the answers to these questions. ISO
9001:2000 specifically requires management review with defined inputs and outputs. And
there's no sense in conducting an ISO 9001 management review, then conducting a
separate review of the organization's performance--they should be the same review. The
more timely and action-oriented the review, the better.

Some of the best approaches to reviewing organizational performance are the most
creative. Many organizations design their reviews across a number of different forums
and time frames, which is a practical and realistic way to approach the process.
Regardless of how the review is configured, the three imperatives include data analysis,
identifying opportunities and taking action on them. Smart organizations treat these three
activities as inseparable.

Here are some related questions:

  Who's involved in reviewing the organization's performance?
 What actions have resulted from these reviews?
 How are records of the reviews generated?
 Are all required inputs and outputs addressed by records?
 How does the rest of the organization learn of actions and decisions that are
determined during reviews?

8. What evidence can you provide of continual improvement?

This question can be asked of everybody in the organization. In organizations that have
developed improvement tools and provided opportunities for their application, this is an
easy question. In organizations where improvement efforts are very narrowly applied, it
becomes a much harder question. There should certainly be some evidence of continual
improvement within the scope of the audit. Strategic improvements are impressive, of
course, but all improvements have value. This question actually summarizes many of the
earlier questions into a single point of inquiry. The ultimate purpose of a management
system is to provide a means for improvement.

Just because one or two people aren't able to provide evidence of improvement isn't
necessarily a problem. It could indicate weak improvement efforts, though, and further
investigation would certainly be warranted. In very mature organizations, all personnel
are involved in making improvements, and proof of this happening is abundant.

These are some related lines of inquiry:

 Who's involved in improvement efforts?

 What tools are used to pursue continual improvement?
 How are personnel trained to use improvement tools?
 How are improvement ideas prioritized?
 How are employees made aware of improvement efforts and successes?

9. How are training needs determined?

This might seem like a strange question on a list of most important audit questions, but
it's very significant. Developing human resources is one of the keys to organizational
success. This audit question attempts to probe the degree of planning that goes into
developing these resources. Is training performed as a knee-jerk activity without any
underlying objectives? Or is it geared toward empowering each employee with the skills
and knowledge needed to propel the organization forward? During the audit, make sure to
probe the training needs that have been determined for all levels of personnel: hourly,
salaried, temporary, highly skilled technicians and top management. Training is an
activity that applies to all personnel, not just a narrow slice of the organization.

Here are some related questions:

 What kind of orientation training is provided when employees are hired?

 How are personnel made aware of the organization's mission, values and
measurable objectives?
  How is the effectiveness of training evaluated?
 What happens when training is determined to have been ineffective?
 What records of training are maintained?

10. What's the most important thing about your job?

This is an exploratory question aimed at assessing the degree of planning that went into
developing the management system. The answer can be compared to the formal controls
in place (e.g., documentation, training, verifications, data analysis, etc.) to determine how
deliberately the management system was designed and implemented. If you learn that
the most important thing about the job is receiving timely and complete feedback from
the downstream department, then it will be revealing to explore if the feedback exists
and what's done with it.

This question requires a great deal of skill on the auditor's part because the information
may or may not lead to any logical conclusions. The auditor must have the experience
and maturity to know when an issue is worth exploring in detail. In other words, don't
allow this question to become an endless fishing expedition. Explore the important
elements of a job, compare what you learn against the controls in place and cross-check
the facts with other personnel doing similar jobs. It can produce powerful insights.

Consider these related questions:

 What's the hardest thing about your job?

 What are some things you'd like to change about your job?
 What resource would help you be more effective?
 What should your manager know that he or she currently doesn't know?
 If you were the manager here, what would you do differently?

The 10 questions presented here represent only a slice of what might matter to a typical
organization. You will want to refine this list based on special concerns and risks faced by
your company. Figure out what matters most to your organization and focus your audit
process on those things. There isn't enough time and energy to focus on everything. An
audit process that monitors the organization's key success factors will always be relevant
and always produce powerful results.