Chapter 9

Testing and
evaluation of
systems

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
 Learning objectives

• To suggest audit and systems objectives for selected components of the

financial statements.
• To evaluate systems in use in selected areas and draw up audit conclusions.
• Explain the role of tests of control, and in particular those used to test
computer systems.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 2
Case studies: setting system objectives, use of flow charts, identification of
key questions, and evaluation of systems (1)

• Case study 9.1 Broomfield plc: integrated computerized sales, trade receivables, cash and
inventory system.

1. What do you think are the objectives of this system? Phrase them as audit objectives.
2. Identify control points in this system and suggest controls that should be in force.
3. Refer to the suggested key and subsidiary questions for receipt of sales order and credit
control and decide how you would evaluate the parts of the system relating to these
aspects.
4. There are at least two matters that might affect the inventory figure appearing in the
financial statements. Identify these matters and suggest how the company might overcome
them.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 3
Setting system objectives, use of flow charts, identification of key
questions and evaluation of systems (2)

• Case study 9.2 Troston Limited: integrated production and

production wages system.

1. State the broad objectives of the wages system.

2. List the key questions in the wages area.
3. Review the payroll system of Troston plc and comment on the
strengths and weaknesses you have identified.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 4
Setting system objectives, use of flow charts, identification of key
questions, and evaluation of systems (3)
• Case study 9.3 Broomfield plc: part of computerized purchases and trade creditors
system.
1. State the broad systems objective of a purchases and related trade payables
system.
2. Redraft the part of the flow chart as Ivor does not always send out an order on
the basis of the requisition and has the authority to change inventory reorder
levels.
3. Suggest key questions in the following areas
1. Requisitioning
2. Purchasing
3. Receipt of goods and services
4. Processing of suppliers invoices
5. Entry in purchases ledger
6. Entry in general ledger
4. Evaluate the system you have recorded.
Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 5
Setting system objectives, use of flow charts, identification of key
questions, and evaluation of systems (4)

• Case study 9.4 Burbage Limited: sales order processing system and
identification of application controls
• Analyse the Burbage Limited sales order processing system and
identify areas of significant weakness in internal control and
suggest steps the company should take to rectify any weaknesses.
Your answer should be framed under:
– application controls:
(a) input (b) processing (c) output
– other matters

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 6
 Case studies: setting
system objectives, use
of flow charts,
identification of key
questions and
evaluation of systems
(5)

Figure 9.1

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 7
 Case studies:
setting system
objectives, use of
flow charts,
identification of key
questions and
evaluation of
systems (6)

Figure 9.2

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 88
 Case studies: setting system objectives, use of flow charts, identification of
key questions, and evaluation of systems (7)
Figure 9.3
A. Suppliers’ records are updated using a terminal in the buying
department.
B. The purchases budget is prepared by Ivor Jordan and stock
reorder levels set (annually).
C. When reorder level is reached the computer automatically
prepares purchase requisitions which form the basis for
preparation of the purchase order by Ivor Jordan.
D. There is formal transfer of purchase orders and control totals
pre-list and control logs are signed by both Jordan and Owler.
E. The exception report is used by Owler to make corrections in a
separate purchase order run.
F. Copy 4 of the official purchase order becomes a goods received
note, that is passed to Janet Black for matching with purchase
invoice. She enters supplier and general ledger codes on the face
of the invoice.
G. There is a formal transfer of purchase invoices and control totals
pre-list and control logs are signed by both Black and Owler.
H. The exception report is used by Owler to make corrections in a
separate stock/creditors update run.
I. The chief accountant reviews monthly purchases and creditors
listings for reasonableness and signs to indicate approval.
Likewise he reviews the information for reasonableness.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 99
 Case studies:
setting system
objectives, use of
flow charts,
identification of
key questions,
and evaluation of
systems

Figure 9.4

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 10
Definitions: walk through test, tests of controls, substantive tests

Table 9.1

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 11
11
 Tests of controls and substantive testing (1) - Para A4 ISA 330

• ‘The auditor’s assessment of the identified risks at the assertion level provides a basis for
considering the appropriate audit approach for designing and performing further audit
procedures. For example, the auditor may determine that:
(a) Only by performing tests of controls may the auditor achieve an effective response to
the assessed risk of material misstatement for a particular assertion;
(b) Performing only substantive procedures is appropriate for particular assertions and
therefore, the auditor excludes the effect of controls from the relevant risk assessment.
This may be because the auditor’s risk assessment procedures have not identified any
effective controls relevant to the assertion, or because testing controls would be inefficient
and therefore the auditor does not intend to rely on the operating effectiveness of controls
in determining the nature, timing and extent of substantive procedures; or
(c) A combined approach using both tests of controls and substantive procedures is an
effective approach.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 12
 Tests of controls and substantive testing (2)
• Para 18 ISA 330:
‘Irrespective of the assessed risks of material misstatement, the auditor shall design
and perform substantive procedures for each material class of transactions, account
balance, and disclosure.’
• Para A43:
‘Depending on the circumstances, the auditor may determine that:
– Performing only substantive analytical procedures will be sufficient to reduce audit
risk to an acceptably low level. For example, where the auditor’s assessment of risk
is supported by audit evidence from tests of controls.
– Only tests of details (a substantive procedure) are appropriate.
– A combination of substantive analytical procedures and tests of details are most
responsive to the assessed risks.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 13
 Important note on recording systems, tests of control and
substantive procedures S1

• Distinguish between:
1. tests recording systems and
2. testing controls and
3. substantive tests of detail.

• Objective of tests differ, but actual procedures may be similar.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 14
 Important note on recording systems, tests of control and
substantive procedures S2
1. Determine system and controls in place - walk-through tests - tracing a few transactions
through financial reporting system.
• Select sales order from credit customer and trace it to:
• (a) granting credit decision (b) despatch note (c) sales invoice (d) entry in trade receivables
account (e) entry in inventory movement record.
• Objective: NOT to prove ALL transactions properly recorded but to understand system, record
it, and see if entity has appropriate controls.
• Para A13 ISA 315 (walk through tests in existing client): ‘The auditor is required to determine
whether information obtained in prior periods remains relevant, if the auditor intends to use
that information for the purposes of the current audit. This is because changes in the control
environment, for example, may affect the relevance of information obtained in the prior year.
To determine whether changes have occurred that may affect the relevance of such
information, the auditor may make inquiries and perform other appropriate audit procedures,
such as walk-throughs of relevant systems.’

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 15
 Important note on recording systems, tests of control and
substantive procedures S3
2. Auditors have to decide whether the system appears strong enough
for them to rely on it in arriving at conclusions.
• Auditors perform tests of control to satisfy themselves initial conclusion
about system is valid. Auditors might select 20 sales despatch notes and
trace in the same way as for walk-through test.
• Objective: to enable them to decide if they can in fact rely upon the
system and controls.
• Note: an important element is to ensure the transactions selected have
been authorized by someone in authority.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 16
 Important note on recording systems, tests of control and
substantive procedures S4
3. Auditors decide on level of control risk, leading to decision on level of substantive procedures
to enable conclusion at assertion level on validity of transactions and balances in population.
(a) if control risk LOW, perform analytical review – do figures make sense in the light of what
is known about the company. Means auditor would rely on substantive analytical procedures.
(b) If control risk HIGH, auditor may decide NOT to rely on controls and use tests of detail
only, selecting (say) 100 sales despatch notes and tracing to other records = substantive tests
of detail.
(c) If system WEAK, but auditors decide to RELY ON IT TO SOME EXTENT, might select a
combination of substantive analytical procedures and tests of detail.
• Tests of controls and substantive tests of detail may be carried out at the same time = dual
purpose test.
• Sometimes only tests of controls will give needed audit satisfaction because effective
substantive tests cannot be designed – as in complex systems.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 17
Relationship: walk through tests, tests of controls and substantive
testing Figure 9.5

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 18
18
 Activity 9.1
• Rummond Limited is a trading company and performs inventory count at end of year
to be satisfied:
1. Quantities of recorded inventories are correct.
2. Physical condition is such that they are saleable.
3. Correct cut-off.
a) examine the inventory count instructions
b) at count auditor observes count procedures being carried out by Rummond staff,
testing a few of the items counted themselves
c) select items counted at random and record in working files for comparison later with
quantities in inventory valuation sheets.
• Refer to important note on recording systems, tests of control and substantive testing
and explain what the audit objectives are at each stage, (a), (b) and (c) above.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 19
 Examples of tests of controls
• Tests of information/audit trail.
• Testing outputs on a restricted basis.
• Interviews with company staff (inquiry) using interviewing style conducive to
getting people to be open with them.
• Observing staff at work (observation), keeping eyes open and not assuming
staff will always operate in the manner they have told you they do.
• Re-performance of control procedures.
• Examination of management reviews.
• Testing reliability of budgets prepared by management.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 20
 Approaches to computer systems

• Auditing round the computer

• Auditing through the computer: computer assisted
audit techniques (CAATs)
• Auditing with the computer

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 21
 Specific tests of control in computer
systems
• Code reviews of programs.
• Use of test data.
• Use of program code comparison.
• Continuous review of data and its processing.
• Integrated Test Facility (ITF).
• Systems Control and Review File (SCARF).

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 22
 Evaluation of systems and audit conclusions
– Activity 9.2 (1)

• Weaknesses highlighted: Case study 9.3 (Broomfield)

A. No responsible official reviews minimum inventory levels. Management authorises Ivor Jordan to disregard
requisition if he believes goods not required AND gives him authority to change minimum inventory levels
on master file. Weakness: Ivor could circumvent management's initial budget decision.
B. Ivor Jordan does not check official PO to original PO.
C. Eric Owler wholly responsible for investigating and correcting errors in exception report – Ivor should be
involved. Ivor should compare official POs with initial orders as risk official orders incorrect. Main store
does not get copy of initial order.
D. Janet Black has no access to current prices and terms – no independent check. Too much power with Ivor
Jordan.
E. Insufficient independent checking of exception report and invoice listing.
F. Janet Black codes invoices, but no one checks her work before processing. Strength is that chief
accountant reviews cost information for reasonableness.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 23
 Evaluation of systems and audit conclusions
– Activity 9.2 (2)
• Case study 9.3 (Broomfield): suggest in broad terms what you might do as a result of
weaknesses found in the purchases system.
• The auditor might perform tests in the following areas:
– Review exception reports to ensure appropriate corrective action taken.
– Check coding of purchase invoices to ensure costs have been charged to appropriate cost
and expense accounts. Support this work by analytical reviews of costs.
– Check interventions to amend inventory master files using information/audit trail as an aid.
– Match purchase orders and purchase invoices to ensure prices and terms are in agreement.
– Check on reasonability of prices charged by suppliers.
• Some of this work might be carried out by internal auditors, in which case the work of the
internal auditors should be reviewed by the external auditors.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015 24
Figure 9.1 Data flow diagram for Bloomfield plc sales and
trade receivables system

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Figure 9.2 Computer systems flowchart for production
payroll system of Troston plc

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Figure 9.3 Document flowchart for the purchases system
of Broomfield plc
A. Suppliers’ records are updated using a terminal in the
buying department.
B. The purchases budget is prepared by Ivor Jordan and stock
reorder levels set (annually).
C. When reorder level is reached the computer automatically
prepares purchase requisitions which form the basis for
preparation of the purchase order by Ivor Jordan.
D. There is formal transfer of purchase orders and control
totals pre-list and control logs are signed by both Jordan
and Owler.
E. The exception report is used by Owler to make corrections
in a separate purchase order run.
F. Copy 4 of the official purchase order becomes a goods
received note, that is passed to Janet Black for matching
with purchase invoice. She enters supplier and general
ledger codes on the face of the invoice.
G. There is a formal transfer of purchase invoices and control
totals pre-list and control logs are signed by both Black and
Owler.
H. The exception report is used by Owler to make corrections
in a separate stock/creditors update run.
I. The chief accountant reviews monthly purchases and
creditors listings for reasonableness and signs to indicate
approval. Likewise he reviews the information for
reasonableness.

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Figure 9.4 Sales order processing (Burbage
Limited)

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Figure 9.5 Walk-through tests, tests of control and substantive
procedures: conclusions, decisions and extent of tests and
procedures

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015
Figure 9.6 Auditing round and through the computer

Use with The Audit Process: Principles, Practice and Cases, 6th edn
ISBN 978-1-4080-8170-9 © Iain Gray, Stuart Manson and Louise Crawford, 2015