Chapter 4

TESTING – GENERAL AND

AUTOMATED CONTROLS
Mentor: TS. Truong Tuan Anh
Presenter: Huynh Thi Nan
Content
1. Introduction to Basics of Testing (Reasons for Testing)
2. Audit Planning
3. Audit Testing
4. Testing Critical Control Point
5. Test Effectiveness of IS controls
6. Tests of General Controls at the Entitywide and
System Levels

2
Content (cont)
7. Tests of General Controls at the Business Process –
Application Level.
8. Tests of Business Process Application Controls & User
Controls
9. Appropriateness of Control test
10. Multi year testing plan
11. Documentation of control testing phase

3
Content (cont)
12. Audit Reporting
13. Concurrent or Continuous Audit & Embedded Audit
Modules 
14. Hardware Testing 
15. Operating system review 
16. Network Review 

4
1. Testing: Definition
● Testing is the process of assessing Correctness, Completeness and
Quality of system.
● Testing is a process to determining whether the controls are
adequately protect the system.
● Testing of the controls design and the reliable results are done by one
of the following methods:
 Substantive Testing
 Compliance Testing

5
1. Testing: Definition (cont.)
● The Information System (IS) controls audit involves the
following three phases:
 Planning : Here auditor determines the way to collect the
evidence to achieve objectives IS audit.
 Testing : Here auditor tests the effectiveness of IS controls.
 Reporting : Here auditor concludes and reports the result of
audit to the management.

6
2. Audit Planning
● Planning occurs throughout the audit and includes the following
activities –
 Obtain an understanding of the entity and its operations.
 Obtain an understanding of internal controls.
 Assess the risk.
 Design the nature, extent and timing of audit procedure.

7
2. Audit Planning (cont.)
● In planning the IS controls audit, Auditor uses the concept of
Materiality and Significance.
● According to these concepts –
Auditor is not required to spend resources on item that are not
material and significance i.e. those that would not affect the
judgment of users of audit report.

8
3. Audit Testing
Some important decisions before testing begins ~
● Testing Methodology-
 Auditor must find testing methods to determine that controls
are effective. This may include reviewing documentary
evidence, conducting personnel interview and personal
observation.
● File interrogation-
 Auditor must browse directories of PC to investigate use developed
application files.

9
3. Audit Testing (tt)
● Test pack-
 Auditor uses valid and invalid data to test the ability to
prevent, detect, and correct errors.
 The intensity and extent of testing depends upon importance of
the application.
● Automated tools –
 Audit team can use GAS (Generalised Audit Software) to do
sampling, data extraction, summarizing and reporting.

10
3.0 Types of Control Audit Test
● Financial Audit -
If IS control audit is performed as a part of financial audit, the
auditor understand the controls over financial reporting to assess
the risk of misrepresentation.
● Performance Audit -
If IS control audit is performed as a part of performance audit, the
auditor should evaluate the design and operating effectiveness of
all the controls.

11
 3.1. IS Controls Audit Process
● Obtaining an understanding of an entity and its operations and key business
processes,
● Obtaining a general understanding of the structure of the entity’s networks,
● Identifying key areas of audit interest (files, applications, systems, locations)
● Assessing IS risk on a preliminary basis,
● Identifying critical control points (for example, external access points to
networks),
● Obtaining a preliminary understanding of IS controls, and Performing other
audit planning procedures.

12
3.2 Identify Key Areas of Audit Interest
● Key areas are those applications and files that are critical in
achieving audit objectives.
 For financial audit : key financial applications
 For performance audit : all key system applications
● For each key area, auditor should document –
 Operational location
 Significant components (h/w, s/w)
 Other support systems/resources
 Prior audit reports

13
 3.3 Performing Information System Controls
Audit Tests

The auditor identifies control techniques and determines the effectiveness

of controls at each of the following levels:
● Entity wide or Component Level: (General controls)
● System level (General controls)
● Business process application level

14
3.3 Performing
Information
System Controls
Audit Tests
(cont.)

15
 4. Testing Critical Control Point
● Critical control point is that component of system which is of
significant importance.
● Auditor tests controls related to the component, its operating system
and its applications.
● For e.g. – Router. Auditor tests the control related to the router itself,
its operating system and applications.

16
 5. Test Effectiveness of IS controls
● Auditor should conduct test of those control technique that are
effective in operation.
● To do so the best way is to test control in tired basis. Starting with –
 Entity wide controls
 System level controls
 Business process application level controls
 Data management controls
● Ineffective IS controls at each tier generally prevent effective control
at the subsequent tier.

17
 6. Tests of General Controls at the Entitywide
and System Levels
● General controls at entity wide and system level can be tested using
techniques such as Inquiry, Observation, Inspection and re-performance
thru test software.
● After reaching favorable conclusion on general controls at these level auditor
test general control at business process application level.
● If general control are not effectively operating then auditor should-
 Determine nature and extent of risk resulting from ineffectiveness.
 Identify and test any manual controls as compensating control.

18
7. Tests of General Controls at the Business
Process – Application Level

● These business process application level general controls are referred

to as Application Security (AS) controls.
● If general controls are not operating effectively within the business
process application, business process application controls and user
controls generally will be ineffective.

19
 7. Tests of General Controls at the Business
Process – Application Level(cont.)
● If the IS controls audit is part of a financial or performance audit, the
IS controls specialist should discuss the nature and extent of risks
resulting from ineffective general controls with the audit team.
● The auditor should determine whether to proceed with the evaluation
of business process application controls and user controls.

20
8. Tests of Business Process Application
Controls & User Controls
● The auditor generally should perform tests of those business process
application controls (business process, interface, data management),
and user controls necessary to achieve the control objectives.

21
 8. Tests of Business Process Application
Controls & User Controls (cont.)

● If IS controls are not likely to be effective, the auditor should obtain a

sufficient understanding of control risks arising from information
systems to:
 identify the impact on the audit objectives,
 design audit procedures,
 and develop appropriate findings.

22
8. Tests of Business Process Application
Controls & User Controls (cont.)
● The auditor considers whether manual controls achieve the control
objectives, including manual controls that may mitigate weaknesses
in IS controls.
● If IS controls are not likely to be effective and if manual controls do
not achieve the control objectives, the auditor should identify and
evaluate any specific IS controls that are designed to achieve the
control objectives to develop recommendations for improving internal
controls.

23
9. Appropriateness of Control test
● To keep appropriateness of control tests the auditor should perform
appropriate mix of audit procedure that includes the following –
 Inquiries of IT and management personnel
 Questionnaires
 Review documentation of control procedures
 Inspection of approvals(authorisation)
 Analysis of system information(configuration)
 Analysis of output (accuracy of processing)
 Review of data file
 Re-performance of the control (use of test data)

24
10. Multi year testing plan
● Where auditor regularly performs control audit of the entity, the
auditor may develop a multi year plan for control audit.
● These plan should cover not more then 3 years and include schedule
and scope of assessment.
● Under multi year plan each control is tested at least once during the
multi year period.
● This concept allow auditor to test controls on risk basis rather then
testing every control every year.

25
10. Multi year testing plan (cont.)
● For example a multi year plan for an entity with 7 applications might
include comprehensive test of 2-3 application annually.
● Multi year plans are not appropriate in all situations. For example –
 They are appropriate for first time audit.
 They are not appropriate where audit has not been tested within a
recent period.
 For entity that do not have strong entity wide controls.

26
11. Documentation of control testing phase
● Information gathered during testing phase should be documented. This
include –
 Understanding of IS.
 IS control objectives and activities.
 Description of control techniques used by entity.
 Specific test performed.
 Description of nature, extent and timing of test.
 Evidence of effectiveness of controls.
 If ineffective then compensating controls.
 Auditor’s conclusion about effectiveness of controls.
 For each weakness; material weakness, significant deficiency or just
deficiency.

27
12. Audit Reporting
● After completing testing auditor summarizes the audit result, draw
conclusion on the control weakness.
● Auditor prepares this report on entitywide, system and BPA level
collectively.
● Such documentation may be developed as the audit progresses, allowing
auditor to demo. that the weakness exist and can be exploited.
● Auditor should also document the potential impact of weakness on
completeness, accuracy, validity, confidentiality of system.

28
Some audit terms
Substantive testing
● Substantive testing is used to determine the accuracy of information
generated by a process.
● Auditor generate and process test data to verify the processing steps.
● Where controls are evaluated as ineffective, substantive testing may be
required.
● Auditor uses CAAT to generate test pack and conduct the test.

29
Some audit terms (cont.)
Analysis
● Interviews and tests provide the raw facts for drafting a audit report
but does not guarantee to produce a quality audit report.
● Analysis is important to convert this raw material into finished
product.
● Timely analysis gives the auditor time to conduct further test and
allow more time for corrective actions.
● Thorough analysis includes the following 4 steps –

30
Steps
Re-
examinatio
n

Cause of
conclusion deviation

Materiality
Exposure

31
 Steps
Step 1 : Re-examination
● The two factors to be re-examined are : Standard and Facts.
● Standard are the rules, procedures and practices that defines how an
operation under audit should function.
● The standards must be clearly understood by the auditors, because wrong
understanding leads to incorrect findings.
● Facts are evaluated after standards are reviewed. For accuracy the sample
should be
 Large enough to reflect behavior of population.
 Representative of current control activity

32
 Steps (cont.)
Step 2 : Cause of Deviation
● After understanding standards and facts, auditor identify the causes of the
deviation.
● Determining the cause is like answering the following questions –
 Who (responsible)
 What (initiating event)
 Where (system component)
 Why (contributing factor)
 When (timing)
● Cause determination helps to identify exposure and formulating
recommendations.
33
 Steps (cont.)
Step 3 : Exposure and Materiality
● These are consequences of deviation.
● Exposure is the potential loss, harm, damage, theft or inefficient use
and
● Materiality is a qualitative judgment about whether a deviation’s
frequency of occurrence and degree of exposure are significant enough for
the deviation to be corrected.
● Degree of exposure is related to Proximity and Severity of risk.
● Proximity refers to the extent of asset availability to the users or
environment. Limited access – less proximity.

34
Steps (cont.)
Step 4 : Conclusion
● Conclusions are auditor’s opinion on, whether the audit subject
area meets the audit objectives.
● Conclusions must be supported by factual evidences.

35
13. Concurrent or Continuous Audit &
Embedded Audit Modules
Snapshot
s
● Many audit tools are also available
some of which are described: CIS ITF

SCARF

Created By Manish Mathur

36
13. 1 Snapshot
● Special audit module built into the system where transaction
processing occurs.
● It takes images of the transactions of audit significance and stores
them in auditor’s file.
● Main issues to decide are –
 Location of snapshot
 Condition to capture the image
 Reporting system

37
13.2 Integrated Test Facility
● It involves creation of dummy entity in the client system and
processing special audit data against that.
● Methods of creation of test pack –
 An embed audit module, recognize transaction having certain
characteristic. These tagged tr. can be used as test pack.
 Auditor may use test data specially prepared for audit.

38
13.2 Integrated Test Facility (cont.)

● Method of removing effect of ITF Tr. –

 Application system may be programmed to recognize ITF Tr. and
ignore them in reporting.
 Auditor may submitting additional inputs that reverses the effect
of ITF Tr.

39
 13.3 System Control Audit Review File
● It involves use of special audit module within system under audit.
● It provide continuous monitoring of system’s transactions.
● Collected information is stored in special audit file : SCARF.
 Application system errors
 Policy and procedure variance
 System exceptions
 Statistical sample
 Snapshot and extended records
 Profiling data
 Performance measurement

40
 13.4 Continuous and Intermittent Simulation (CIS):
● The database management system reads an application system
transaction. It is passed to CIS. CIS then determines whether it wants to
examine the transaction further. If yes, the next steps are performed or
otherwise it waits to receive further data from the database management
system.
● CIS replicates or simulates the application system processing.
● Every update to the database that arises from processing the selected
transaction will be checked by CIS to determine whether discrepancies
exist between the results it produces and those the application system
produces.

41
13.4 Continuous and Intermittent Simulation
(CIS) (cont.):
● Exceptions identified by CIS are written to a exception log file.
● The advantage of CIS is that it does not require modifications to the
application system and yet provides an online auditing capability.

42
 14. Hardware Testing
● H/w testing is done against FRS and SRS.
● Types –

Function testing Security testing

User interface testing Capacity testing
Usability testing Performance testing
Compatibility testing Reliability testing
Model based testing Installation testing
Error exist testing Maintenance testing
User help testing Accessibility testing
43
15. Operating system review
● Interview IS manager, system programming manager and others
regarding –
 Process of option selection.
 Test procedure for system software
 Review and approval procedure fro test results.
 Implementation procedure
 Documentation requirement
● Review the feasibility study
 Same selection criteria are applied to all proposals.

44
 Operating system review (cont.)
● Review cost benefit analysis –
 Direct financial cost of the product.
 Cost of product maintenance.
 Hardware capacity requirement.
 Training and support requirement.
 Impact of the product on the processing.
 Impact on data security.
 Financial stability of the vendor.

45
Operating system review (cont.)
● Review control over installation of changed System software –
 All updates are implemented.
 Installation of changes SS is scheduled when they least impact
processing.
 There is a written plan for testing.
 Problems encountered during testing were resolved and changes
were re-tested.
 Test procedures ensure that changes do not create new problems.
 Restoration procedure are in place.

46
Operating system review (cont.)
● Review control over installation of changed System software – (cont.)
 Software must be properly authorised prior moving from test to
production environment.
 Access to libraries is limited to individual’s need.
● Review system software’s maintenance activities
 Changes made to the SS are documented.
 Vendor support current version of software.

47
 Operating system review (cont.)
● Review SS documentation –
 Installation control statement.
 Parameter tables.
 Exit definition.
 Activity log.
● Review SS for adequacy of controls, such as –
 Change procedure controls
 Authorisation controls
 Access privileges controls
 Documentation controls
 Testing controls
 Audit trails

48
Operating system review (cont.)
● Review authorization document to determine –
 Addition, deletion or change to access authorisation is documented.
 Attempted violation reporting and response is documented.
● Review SS security, to determine –
 Procedure have been established to prevent bypass of access control.
 Procedure have been established to limit access to system interrupt
capability.
 Physical and logical access controls are adequate.
 Vendor supplied passwords are changed.

49
Operating system review (cont.)
● Review database supported controls –
 Access to shared data is appropriate.
 Data organization is appropriate.
 Change procedures are established to ensure integrity of DBMS.
 Integrity of data dictionary is maintained.
 Data redundancy is minimised.

50
16. Network Review
● Review the LAN, to understand –
 LAN Architecture
 Cost benefit analysis
 LAN topology
 LAN components
 Internetworking
 LAN uses
 LAN administrator
 LAN users

51
 Network Review (cont.)
● Review LAN to make an assessment of –
Threat Impact Controls
● Review physical access controls –
 Ensure that LAN h/w, file server and documentation are located in
secured area.
 Verify that LAN wiring is physically secured.
 Observe LAN file server and verify that it is secure.
 Keys to file server facility is controlled.
 Obtain copy of key log for the file server room and determine that keys
are assigned to appropriate persons.
 Select keys held by people and determine that these keys do not permit
to access LAN facilities.
52
Network Review (cont.)
● RReview Environment controls to –
 Ensure that LAN file server is protected from electric surges.
 Ensure that AC and humidity control system are adequate to
maintain temperature.
 Ensure that LAN server is equipped with UPS.
 LAN file server is free of dust, smoke and pollutants.
 Backup disks are protected from environmental damage.
 Fire extinguishers are nearby.
 Food and beverages are prohibited.

53
 Network Review (cont.)
● Review Logical access controls to ensure –
 Users have unique password, password are change periodically and does
not appears on screen while entry.
 LAN access should be based on written authorization.
 Remote access to the system supervisor should be prohibited.
 All log-on attempts should be logged.
 LAN supervisor should maintain up-to-date information of all outside
communication.
 Evaluate LAN server access profile.
 Attempt to gain access using unauthorised ID/PWD.
 If LAN is connected to an outside source through a modem attempt to
gain access to the LAN thru correct and incorrect means.

54
THANKS YOU !

55