Auditor’s Guide to IT Auditing, Second Edition

By Richard E. Cascarino
Copyright © 2012 by Richard E. Cascarino

15 CHAPTER FIFTEEN

Governance Techniques

T
H IS C H A P T ER COV ER S the need for, and use of, techniques such as change-
control reviews, operational reviews, and International Standards Organization
(ISO) 9000 reviews.

CHANGE CONTROL

Periodically the necessity arises to modify an existing hardware and/or software con-
figuration as a result of:

▪ Hardware changes as a result of performance improvements or reconfigurations

caused by changes to other systems
▪ Hardware failures during normal operations
▪ The detection of a software error during normal operations
▪ Changes to legislation affecting the organization’s business systems
▪ A change to the business operation of the organization requiring alterations within
the information systems

As a result of these changes in the environment, the extent of change required

within the existing system configuration must be determined and the change applied
in a controlled manner so as to avoid any undue disruption to normal processing. It is

179

c15.indd 179 1/31/2012 8:41:39 AM

 180 ◾  Information Technology Governance

critical that during periods of change, the production versions of software are protected
against unauthorized changes, untested changes, or even malicious changes.
Change control’s objective is to ensure risk is controlled, not introduced, during a
change. This means ensuring that:

▪▪ All changes are authorized

▪▪ All authorized changes are made
▪▪ Only authorized changes are made
▪▪ All changes are as specified
▪▪ All changes are cost effective

This control requires a coordinated effort involving managers, users, information

systems personnel, and IT auditors. An effective methodology for authorizing, testing,
and implementing the change in a controlled manner is a prerequisite. In most orga-
nizations this will involve the use of a Change Control Committee involving members of
all of the aforementioned disciplines. This committee is normally involved in evaluat-
ing change requests for corporate or control im­plications, authorizing those requests,
ensuring that testing and documentation of the changes has been carried out, and
finally authorizing the implementation of the change into the live environment.
All requests for amendments to production programs should be made in writing
and include the business justification for the change requested. A full appraisal of the
impacts, justification, and alternatives considered should be undertaken with more
significant changes being subject to more stringent checks. If the change is of a major
nature, a feasibility study may be required.
Once changes are approved by the committee, the work may be undertaken by
such resources as the committee approves (normally IT with some user involvement)
and, once the programmer involved is satisfied that the amended software is working
as intended, independent testing should take place with the user participation prior to
implementation in the production environment. Users, IT staff, and auditors should all
sign off on the change to ensure that their individual needs have been satisfied. It should
be stressed that when the Information Systems (IS) auditor signs off on a change, this
is an indication that audit’s control requirements have been met within the change
system. It is not an indication of quality assurance of the system because this is the
responsibility of both the users and IT management.
All changes to systems, whether hardware, software, or both, should be fully docu-
mented with the effects of all maintenance changes so that subsequent work on the rela-
tive systems can be expedited. With all possible care being taken there is still a chance
that a change to a system will result in a production system failure. As such, operations
recoverability procedures should be in place in the event of a system failure in the new
configuration. This would typically involve securing the condition of the system and
data prior to the change being implemented so that an unsuccessful change can be
appropriately backed-out.
Within a mainframe environment it is common to separate the production and
development versions of programs using completely segregated software libraries. Once

c15.indd 180 1/31/2012 8:41:39 AM

 Governance Techniques ◾ 181

implementation has been authorized, the change controller will normally copy the
amended source code into the production library. For this to be effective, access to pro-
duction libraries must be restricted to the change controller only. This access control is
intended to prevent both accidental and malicious amendments to production software
occurring without appropriate authorization.
Updating software on personal computers and local area networks would appear
more straightforward because they normally involve installation of purchased pack-
ages. Unfortunately, not all purchased packages function immediately as intended.
In the smaller environment of personal computers it is common that backups are not
taken prior to system changes and that the introduction of a new version of software
or even new software altogether may result in significant damage to the production
environment.
Personal computers and local area networks also require careful control over
changes made. The change-control processes may be different for the surrounding
mainframe computers; nevertheless appropriate change-control procedures must be
implemented.

PROBLEM MANAGEMENT

The changes thus controlled are known and planned changes. The procedures involve
ensuring prior authorization for all changes, supervision of the change process, ade-
quate testing of all changes, and user sign-off on all changes.
Periodically things will go wrong with a system, which necessitates an urgent
repair. Such changes are not known in advance and are commonly executed and per-
mission sought retrospectively. Such changes are controlled using Problem Management.
Problem Management’s objective is to control systems during emergency situations
arising from unforeseen changes. Typically this will involve bypassing normal control
mechanisms and may require direct programmer access to live data. This must be con-
trolled separately and must involve user authorization, even retrospectively.

AUDITING CHANGE CONTROL

From an audit perspective, the IT auditor will seek assurance that change-control pro-
cedures are in place and effective over changes to hardware, software, telecommuni-
cations, or anything that affects the processing environment. Sources of evidence for
the auditor would include minutes of change-control committee meetings, software
movement reports, access-control logs, and system-failure records.
The auditor will typically seek to ensure that:

▪ Requests are recorded and stored for reference

▪ Each change is assessed prior to acceptance of the change, based on its projected
effect to the computer system and business operations

c15.indd 181 1/31/2012 8:41:39 AM

 182 ◾ Information Technology Governance

▪ Unauthorized changes are limited by automated or manual controls

▪ A problem management change process is in place whereby the reasons for emer-
gency changes and the authorization mechanisms are clearly defined
▪ Change documentation is kept up to date with all maintenance tasks and changes
comprehensively recorded
▪ All new software releases pass through change control

OPERATIONAL REVIEWS

Operational auditing involves first determining management’s objectives followed by

establishing which management controls exist leading to effectiveness, efficiency, and
economy. The auditor must determine which key performance indicators are in use
and their appropriateness as well as determining the achievement of control objectives.
The term operational audit is commonly used to cover a variety of audit types. An
operational audit may cover the evaluation of some or all of the following:

▪ Internal controls
▪ Compliance with laws, regulations, and company policies
▪ Reliability and integrity of financial and operating information
▪ Effective and efficient use of resources

Operational auditors require standards against which current operations may be

compared and evaluated. It is management’s responsibility to devise and use appropriate
standards to evaluate operating activities, and operational auditors will usually start
with criteria that have been established by management (performance standards) or by
some oversight board or agency.
In the absence of standards, the auditors may have to borrow from other sources
or develop some type of criteria against which to compare performance. This is often a
difficult task because frameworks such as Control Objectives for Information and related
Technology (COBIT®) may not have been implemented in a sufficiently detailed man-
ner and auditors should get management’s reaction to the suitability of any criteria so
developed. Reasonable criteria for evaluating performance are absolutely essential for
successful operational auditing because no evaluation of operations is possible without
a standard for comparison. While subjectivity cannot be completely avoided, objective
criteria, which are considered appropriate and reasonable by both the internal auditors
and IT, are essential for continuing success.

PERFORMANCE MEASUREMENT

Performance measurement is a philosophy in which feedback is used to make ongo-

ing adjustments to the course of the organization toward its vision. For example,
the information derived from budgetary or client satisfaction measurements may

c15.indd 182 1/31/2012 8:41:39 AM

 Governance Techniques  ◾ 183

provide the feedback used to assess the effectiveness of an organization from a vari-
ety of viewpoints. Using this feedback, it is possible to ensure continued excellence of
programs and services in response to changes within both the internal and external
environments.
The process commences with the setting of business objectives and the development
of strategies and plans to achieve these objectives within an overall control framework.
This is followed by the development of appropriate performance measures to assess prog-
ress toward the objectives.
Performance-measurement systems provide the feedback information required
to determine if executive management strategies have been effectively converted into
operational decisions.
Performance measurement provides a balanced, methodical attempt to assess the
effectiveness of an organization’s operations from multiple vantage points—financial,
client satisfaction, internal business, and innovation/learning.
The Balanced Scorecard approach can give the auditor well-structured measure-
ment criteria if it has been appropriately implemented. The mechanics of performance
measurement are complex and the development and deployment of the process may be
painful. Typically many measures will be evaluated before a key set will emerge. Many
choices will involve industry best-practice measures so that a competitive benchmark
can be established.
Improving performance measurement involves the development of integrated per-
formance-measurement systems that are built around a strategic theme such as busi-
ness strategy or value creation. They involve measuring those aspects of the IT structure
that relate the activities of people and processes in the IT organization to the intended
outcomes for the IT stakeholders.
Integrated performance measurement systems are a significant improvement
over prior evaluation structures but still do not eliminate some of the basic dif-
ficulties of performance measurement. IT can be a complex organization offering
considerably more opportunities for measurement than management can effectively
employ. The difficulties lie in reducing the required number of measures to a sig-
nificant few.
Managers generally understand how effective measurement provides key support
in the pursuit of organizational goals when the consequences of performance results
are communicated and understood. Within IT they tend to support the concept of per-
formance measurement because their experience has shown it to be effective in helping
to achieve success. Managers who use performance measurement on a regular basis
understand the difficulties inherent in the process. Many measurement criteria form an
imperfect definition of the underlying idea and can result in rewarding “bad” behavior
and punishing “good” behavior.
Most IT managers understand the shortcomings of measurement systems. They
are fully aware that distortions may be introduced through cost and asset allocations.
They recognize that there may be an inclination to measure the things that are easy to
measure, and to avoid measures that are more difficult with the subsequent distortions
this creates.

c15.indd 183 1/31/2012 8:41:39 AM

 184 ◾ Information Technology Governance

ISO 9000 REVIEWS

The International Organization for Standardization (ISO) is the specialized international

agency for standardization, comprising the national standards bodies of 91 countries.
ISO is made up of approximately 180 technical committees, with each technical com-
mittee being responsible for one of many areas of specialization. In 1987, ISO published
the original set of quality assurance standards commonly known as ISO 9000. The ISO
Quality Management and Quality Assurance System Standards provide a set of require-
ments for quality-assurance systems. Compliance with ISO 9000 standards indicates
that a producer has a basic quality assurance system in place.
Increasingly, customers expect organizations to have their quality systems reviewed
and audited to one of the standards of the series. This involves having an accredited
independent third party conduct an on-site audit of the company operations against
the requirements of the appropriate standard.
Reviewing against a company’s implementation of ISO 9000 involves reviewing:

▪ The methodology, including the philosophy, guidelines, policies, responsibilities,

time line, and deliverables.
▪ The project/process, to ensure its compliance with the methodology and to identify
reasons for any deviations.

An important role of such reviews is the establishment of quality objectives and

reviewing of progress toward achieving the objectives and fulfilling the quality policy.
IT quality objectives are established to improve performance and/or the quality system
and thus fulfill the quality policy and other organizational goals and aspirations. This
ties in closely to COBIT’s use of control objectives.
Enacted in 1994, ISO 9001: 2000 Quality Management Systems replaces the older
ISO 9000, 9001, 9002, and 9003. These standards apply within every organization
and can be either service or product oriented depending upon the orientation of the
organization.
The standards themselves specify the quality levels desired and the organization
is responsible for conducting a gap analysis in order to identify areas of noncompliance.
This then facilitates the closure of these gaps in order to achieve compliance. At the
organization’s discretion, an external review can be carried out in order to determine
compliance and, if achieved, an ISO certificate is issued and the organization recorded
in the ISO registry. This registration is then valid until the next audit.
From an audit perspective, auditors are concerned with ensuring that the functions
and processes of IT achieve their anticipated outcomes and that documented proof of
this exists. Of critical importance to the auditor are specific clauses requiring full docu-
mentation of the quality procedures and processes, requiring competence on the part
of personnel carrying out the work subject to quality assurance, and that acquisition
of resources including hardware, software, and outsourced services meets stringent
quality requirements.

c15.indd 184 1/31/2012 8:41:39 AM