Testing Computer Application

Controls
& CAATTs for Testing Controls
 Computer-Assisted Audit Tools and Techniques
• Application Controls • Computer-aided Audit Tools and
• Input Controls Techniques for Testing Controls
• Processing Controls • Test Data Method
• Output Controls • The Integrated Test Facility
• Testing Computer Application • Parallel Simulation
Controls
• Black-Box Approach
• White-Box Approach
TESTING COMPUTER APPLICATION
CONTROLS
• Control testing techniques provide two (2) information about an
application’s processes:
• 1) accuracy
• 2) completeness

• Two (2) approaches in testing application controls:

• (1) the black box (around the computer) approach
• (2) the white box (through the computer) approach.
 TESTING COMPUTER APPLICATION CONTROLS:
Black-Box Approach
• Auditors seek to understand the
functional characteristics of the
application. How?
• 1) by analyzing flowcharts
• 2) by interviewing knowledgeable
personnel in the client’s organization.
• 2) by reconciling input transactions
processed with output results. • Advantage of the black-box approach:
• the application need not be removed from service
• The output results are analyzed to and tested directly.
verify the application’s compliance
• This approach is feasible for testing applications that are
with its functional requirements. relatively simple.
 TESTING COMPUTER APPLICATION CONTROLS:
White-Box Approach
• Relies on an in-depth understanding of the internal logic of the
application being tested.

• Creating test transactions or test data

• Is used in several techniques for testing application logic directly.
• Auditors are able to conduct precise tests, with known variables, and obtain
results that they can compare against objectively calculated results.
 TESTING COMPUTER APPLICATION CONTROLS:
White-Box Approach
Common types of tests of controls under White-box
approach:
• 1) Authenticity tests
• 2) Accuracy tests
• 3) Completeness tests
• 4) Redundancy tests
• 5) Access tests
• 6) Audit trail tests
• 7) Rounding error tests
 TESTING COMPUTER APPLICATION CONTROLS:
White-Box Approach
Common types of tests of controls under
White-box approach:
• 1) Authenticity tests
• user IDs, passwords, valid vendor codes, and authority tables.
• 2) Accuracy tests
• range tests, field tests, and limit tests
• 3) Completeness tests
• field tests, record sequence tests, hash totals, and control totals
 TESTING COMPUTER APPLICATION CONTROLS:
White-Box Approach
Common types of tests of controls under White-box
approach:
• 4) Redundancy tests
• reconciliation of batch totals, record counts, hash totals, and financial control totals
• 5) Access tests
• passwords, authority tables, user defined procedures, data encryption, and inference
controls
• 6) Audit trail tests
• evidence that the application records all transactions in a transaction log, posts data
values to the appropriate accounts, produces complete transaction listings, and generates
error files and reports for all exceptions
 TESTING COMPUTER APPLICATION CONTROLS:
White-Box Approach
Common types of tests of controls under White-
box approach:
• 7) Rounding error tests
• verify the correctness of rounding procedures.
• Rounding errors occur in accounting information when the level of precision used in
the calculation is greater than that used in the reporting.
• Accumulator  technique used to keep track of the rounding differences between
calculated and reported balances.
• Poor accounting for rounding differences can also present an opportunity for fraud:
• Salami fraud  Tend to affect a large number of victims, but the harm to each is immaterial.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS

• Five (5) CAATT approaches on how application controls are tested:

• 1) Test data method
• 2) Base case system evaluation
• 3) Tracing
• 4) Integrated test facility
• 5) Parallel simulation
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Test Data Method
• Test data method
• is used to establish application
integrity.
• processes specially prepared sets of
input data through production
applications that are under review.
• The results of each test are
compared to predetermined
expectations.
• Purpose: to obtain an objective
evaluation of application logic and
control effectiveness.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Test Data Method
• Four (4) procedures the auditor should perform for the
test data method: Computer Auditors
• 1) Obtain a copy of the current version of the application. Operations
• 2) Create test data PrepareTest
Prepare Test
• test transaction files and test master files
Transaction
Transaction Transactions
Transactions
• a complete set of both valid and invalid transactions. TestData
Test Data
AndResults
And Results
• 3) Test transactions
• may enter the system from magnetic tape, disk, or via an input Predetermined results
Computer
Computer & expectations - Prior to
terminal.
• should test every possible input error, logical process, and
Application
Application processing the test data,
the input is manually
irregularity. System
System processed to determine
• 4) Compare the test results with the expected results what the output should
look like.
• Purpose: to determine if the application is functioning Manually
properly. Manually
Computer
Computer Auditor Processed
• This comparison may be performed manually or through Processed
special computer software.
Output
Output Compares Results
Results
• Results from the test run will be in the form of routine output
reports, transaction listings, and error reports.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Test Data Method
• Figure 7.17 lists selected fields for
hypothetical transactions and AR records
prepared by the auditor to test a sales order
processing application.
• Any deviations between the actual results
obtained and those expected by the auditor
may indicate a logic or control problem.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Test Data Method
Three (3) primary advantages of test
data techniques. Disadvantages of Test Data Techniques
• 1) Minimal disruption to the • 1) the auditor cannot be sure that the
firm's operations application being tested is a copy of
the current application used by
• 2) The auditor obtains explicit computer services personnel
evidence concerning application • 2) The auditor cannot be sure that
functions. the application being tested is the
• 3) Auditors need minimal same application used throughout
computer expertise to use this the entire year
method. • 3) Preparation of the test data is
time-consuming
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Base Case System Evaluation
• Base case system evaluation (BCSE)
• A variant of the test data technique, which the set of test data in use is comprehensive
(complete).

• BCSE tests
• are conducted with a set of test transactions containing all possible transaction types.
• Base case
• the consistent and valid results obtained in the process of test transaction through repeated
iterations during systems development testing.

• When subsequent changes to the application occur during maintenance, their effects
are evaluated by comparing current results with base case results.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Tracing
• Tracing  performs an electronic walkthrough
(step by step) of the application’s internal logic.

• Three (3) steps in the tracing procedure:

• 1. The application under review must undergo a
special compilation to activate the trace option.
• 2. Specific transactions or types of transactions
are created as test data.
• 3. The test data transactions are traced through
all processing stages of the program, and a
listing is produced of all programmed
instructions that were executed during the test.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Integrated Test Facility
• ITF (The Integrated Test Facility )
• is an automated technique that enables the auditor
to test an application’s logic and controls during its
normal operation.
• During normal operations, test transactions are merged
into the input stream of regular (production) transactions
and are processed against the files of the dummy company.
• is one or more audit modules designed into the
application during the systems development process.

• ITF databases
• contain “dummy” or test master file records
integrated with legitimate records.
• create a dummy company to which test transactions
are posted.
COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING CONTROLS:
Integrated Test Facility

Computer Operations Auditors

Dummy Entity or
Actual Dummy Company
ITF Center PrepareITF
Prepare ITF
Actual ITF
or Dummy
Transactions
Transactions
Transactions Transactions
Transactions Transactions
AndResults
And Results
Computer
Computer DataFiles
Files
Application
Application
Data
(RegularData)
Data)
(Regular
ITF Data (Test Data
System
System
(ITF audit module
transactions and
routine transactions)
Reports
Reports Reports
Reports Manually
Manually
WithOnly
With Only WithOnly
With Only Auditor Processed
Processed
ActualData
Actual Data ITFData
ITF Data Results
Results
Compare
s
Auditor analyzes ITF results against
expected results
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Integrated Test Facility
Two (2) Advantages of ITF over test
data technique: Disadvantages of ITF
• 1) Supports ongoing monitoring of • The potential for corrupting the data
controls as required by SAS 78. files of the organization with test data.
• 2) Applications can be economically
tested without disrupting the user’s • This problem is remedied in two ways:
operations and without the intervention
of computer services personnel. • (1) adjusting entries may be processed
to remove the effects of ITF from GL
account balances
• Thus, ITF improves the efficiency of the • (2) data files can be scanned by special
audit and increases the reliability of the software that remove the ITF
audit evidence gathered. transactions.
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Parallel Simulation
• Parallel simulation
• requires the auditor to write a program that simulates
key features or processes of the application under review.
• Simulated application
• Is used to reprocess transactions that were previously
processed by the production application.
• The results obtained from the simulation are reconciled
with the results of the original production run.
• can be written in any programming language (fourth-
generation language generators)
• are usually less complex than the production applications
they represent.
• contain only the application processes, calculations, and
controls relevant to specific audit objectives
 COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Parallel Simulation
• Five (5) steps involved in performing parallel
simulation testing by an auditor:
Computer Operations Auditors
• 1. Understands the application under review.
Actual
Actual
• 2. Identifies those processes and controls in the
application. These are the processes to be simulated. Transactions
Transactions
• 3. Creates the simulation using a 4GL or Generalized Computer
Computer Auditor’s
Auditor’s
Audit Software (GAS). Application
Application Simulation
Simulation
• GAS (generalized audit software)  is the tool use by System Program
System Program
auditors to automate various audit tasks. It is the most
popular of computer assisted audit tools and techniques
(CAATTs). Auditor
ActualClient
Client Auditor Auditor
• 4. Runs the simulation program using selected Actual Simulation
Report Compares Simulation
production transactions and master files to produce a Report Report
Report
set of results. The auditor compares the
results of the processing done
• 5. Evaluates and reconciles the test results with the by simulation program with the
production results produced in a previous run. results of the processing done
by the client’s program.
COMPUTER-AIDED AUDIT TOOLS AND TECHNIQUES FOR TESTING
CONTROLS:
Difference Between EAM & GAS
CAATTs for data extraction software fall into two general categories:

1) Embedded audit modules (EAM) 2) Generalized audit software (GAS) packages

• are designed to extract data from • are designed to access data from
specific applications in real time files after processing is completed.
as the applications are processing • can extract data from the files of
the transactions. any system and require no
• are programmed into the additional programming.
application when it is designed. • are extremely flexible in their
• are very structured in terms of ability to access, manipulate, and
what data the auditor can call for. report data to the auditor.
 EMBEDDED AUDIT MODULE (EAM)
• Objective of the EAM (aka continuous auditing):
• To identify important transactions while they are being processed and extract copies of them in real
time.
• EAM
• a specially programmed module
• embedded in a host application to capture predetermined transaction types for subsequent analysis.

• Two (2) uses of EAM:

• 1) for substantive testing technique
• 2) to monitor controls on an ongoing basis as required by SAS 109.
• Transactions can be reviewed for proper authorization, completeness and accuracy of processing, and
correct posting to accounts.
 EMBEDDED AUDIT MODULE
• Host application
• Processes the selected transaction and stores the
copy of the transaction in an audit file for
subsequent review.
• Audit file/Captured transaction
• Is a copy of the transaction processed by the
host application.
• Is made available to the auditor in real time, at
period end, or at any time during the period,
thus significantly reducing the amount of work
the auditor must do to identify significant
transactions for substantive testing.
• EAM approach
• allows selected transactions to be captured
throughout the audit period.
 EMBEDDED AUDIT MODULE:
Disadvantages of EAMs
Operational Inefficiency Verifying EAM Integrity
• EAMs decrease operational • A high level of program
performance. maintenance in host application
• From the user’s point of view
• EAM may create significant overhead,
also requires the EAMs
especially when the amount of testing is embedded within the hosts
extensive. frequent modifications.
• Disadvantage: Reduce the effectiveness
of the EAM as an ongoing audit tool. • The integrity of the EAM directly
• Solution: Design modules that may be
affects the quality of the audit
turned on and off by the auditor. process.
 GENERALIZED AUDIT SOFTWARE
(GAS)
• GAS
• is the most widely used CAATT (computer-assisted audit tool technique) for IS auditing.
• allows auditors to access electronically coded data files and perform various operations on their
contents.
• Example: ACL

• Four (4) factors that made GAS popular:

• (1) GAS languages are easy to use and require little computer background on the part of
the auditor.
• (2) Many GAS products can be used on both mainframe and PC systems.
• (3) Auditors can perform their tests independent of the client’s computer service staff.
• (4) GAS can be used to audit the data stored in most file structures and formats.
 GENERALIZED AUDIT SOFTWARE:
Using GAS to Access Simple Structures
• Flat-file structure
• Is a simple structure that GAS can gain access with a relatively simple process.
 GENERALIZED AUDIT SOFTWARE:
Using GAS to Access Complex Structures
• Complex structures
• such as a hashed file or other form of random file.
• not all GAS products on the market may be capable of
accessing.
• Solution: A special program that will copy the
records from their actual structure to a flat-file
sequential structure (made by System professionals).
• 1) Utility features in DBMS
• can be used to reformat complex structures into flat files
suitable for this purpose.
• 2) ACL’s Open Database Connectivity (ODBC)
interface
• Can be used to solve many of the problems associated
with accessing complex data structures.
 GENERALIZED AUDIT SOFTWARE:
Using GAS to Access Complex Structures
• To illustrate the file flattening process, consider the • A simpler flat-file version of this
complex database structure presented in Figure 8.29. structure is illustrated in Figure 8.30.
• The database structure uses pointers to integrate three • The single flat file presents the three
related files—Customer, Sales Invoice, and Line Item—in
a hierarchical arrangement. record types as a sequential structure
• Extracting audit evidence from a structure of this that can be easily accessed by GAS.
complexity using GAS may be difficult, if not impossible.
 GENERALIZED AUDIT SOFTWARE:
Audit Issues Pertaining to the Creation of Flat Files

Risks: Control
• The auditor must sometimes rely on • Auditors skilled in programming
computer services personnel to produce a
flat file from the complex file structures.
languages may avoid this
• Risk: Data integrity will be compromised
potential pitfall by writing their
by the procedure used to create the flat own data extraction routines.
file.
• For example, if the auditor’s objective is to
confirm AR, certain fraudulent accounts in
the complex structure may be intentionally
omitted from the flat-file copy that is created.
• The sample of confirmations drawn from the
flat file may therefore be unreliable.