Interactive Workshop On Clinicla Audit

'
.

---..
~
------
-~-~~- ---~-.
.
.' I
.i~
.
"

. '
. ,'

. '
:'.
~
.
./...,...
-
'.~-
,.
, ,
. :
'.,
.'. '
y

.
'.
,
I _
Interactive workshop on
Clinical Audit

'. COURSE NOTES'
~
, .
Conducted by:
NATIONAL ACCREDITATION BOARD FOR HOSPITALS
AND HEALTHCARE PROVIDERS (NABH)
2nd Floor, Institution of Engineers Building

2, Bahadur Shah Zafar Marg, New Delhi 110002, India
Telefax: 91-11-2337 9321, 2337 9621

E-mail: dirnabh@gcin.org
Web site: www.gcin.orgj http://nabh.co/main/

Interactive workshop on clinical audit Page 1 of 117

' .
' .
,
'
.. '-
.
'
.., '
'
.

.

'
'
.

.' Contents
51. No.
1.
Topic
Code of Conduct
Page No .
2.
3.
Key Learning Objectives
Schedule
.. 5
4 -,",,-
4.
5.
What is clinical audit?
Stage 1: Preparing for audit and selection of topic

6
23

...... '
'.
6. Stage 2: Selecting criteria 39
.. "
7. Stage 3: Measuring level of performance 57

' .
8. Stage 4: Making improvements 76
9. Stage 5: Sustaining improvement 94
'
. 10.
11.
Case study-1
Case study-2
105
110
. '
.
'
. ,
.
'

'
.

.
'
' .
Code of Conduct
1. The participants are expected to be punctual for the sessions,
.
'
2, The participants are expected to attend every session,
., 3, The use of cell/mobile

may carry their
phones is discouraged,
cell or mobile phone, provided
However, the participants
it is in silent (and/or
vibration) mode so as not to cause disturbance to fellow participants. The

telephonic talk should be outside the programme room, if at all necessary,
under emergency situation .
'.
4. A harmonious and cordial ambiance is necessary for the programme. The
'..
participants are expected to display team spirit.
5. The participants are expected to actively participate in all the sessions so

~. . that the programme becomes interactive .
, '
6. Grievance, if any, shall be communicated

will be binding,
to the Facilitator, whose decision
;.'


.-
Key Learning Objectives
This course is designed to help to develop competence and confidence to carry
out Clinical Audits. At the end of course, participants should be able to achieve
following course objectives:

1. To understand clinical audit process. To help clinicians decide exactly why
-"
they are doing a particular audit and what they want to achieve through
carrying out the audit.
'.
2. To select the right topic for audit.
3. To use evidence of good clinical practice/evidence based medicine in

designing clinical audits.
4. To help clinicians formulate measures of quality based on evidence of good

practice, as the basis for data collection and also to develop data collection

protocols and tools and advise on data collection for clinical audits while
ensuring
addressed.
that patient confidentiality and data protection issues are '.

5. To understand use of statistics for analysing and presenting findings of data
collection and thus help clinicians to analyse causes of problems that are
affecting the quality of care.
'.

6. To help clinicians manage review of clinical audit findings with their

colleagues and be able to prepare clinical audit reports.

--

Interactiveworkshopon clinicalaudit Page4 of 117

.
"
--"
Programme Schedule
TIME
08:30 a.m. to 09:00 a.m.

TOPIC
Registration
09:00 a.m. to 09:30 a.m .

Introduction
participants
of faculty and
09:30 a.m. to 10:00 a.m . Clinical audit - an overview
10:00 a.m. to 11:00 a.m .

Stage 1: Preparing for Audit and
selection of Topic ..
11:00 a.m. to 11:15 a.m .
11:15 a.m. to 12:30 p.m .

TEA BREAK
Stage 2: Selecting criteria
12:30 p.m. to 01:15 p.m.

Stage 3: Measuring
performance
level of
01:15 p.m. to 02:00 p.m. LUNCH
02:00 p.m. to 02:30 p.m.

Stage 3: Measuring
performance
level of
.,
Stage 4 & 5: Making and sustaining
02:30 p.m. to 03:30 p.m .
improvements
03:30 p.m. to 03:45 p.m. TEA BREAK
03:45 p.m. to 04:45 p.m . Case study
Wrap up; Sharing of best practices;
04:45 p.m. to 05:00 p.m .

Feedback

.~
What is Clinical Audit?

Introduction
"Clinical Audit is a quality improvement process that seeks to improve patient care and

outcomes through systematic review of care against explicit criteria and the implementation
of change. Aspects of the structure,
systematically evaluated against
processes and outcomes
explicit criteria. Where
of care are selected and
indicated, changes are

implemented at an individual, team, or service level and further monitoring is used to

confirm improvement in healthcare delivery"
Principles for Best Practice in Clinical Audit (2002, NICE/CHI)

The authors of the Cochrane Review on audit and feedback offer a narrower definition.
them clinical audit is: "The provision of any summary of clinical performance
For
over a

specified period of time. The summary may include data on processes of care (e.g. number
of diagnostic tests ordered), clinical endpoints (e.g. blood pressure readings), and clinical
practice recommendations (proportion of patients managed in line with a

recom mendation)."
The aim of clinical audit is to measure the gap between ideal practice (determined from

evidence and guidelines) and actual practice. Audit does not seek to apportion blame on n
individual practitioners, but aims to improve the systems in which individuals work. Done
correctly, audit can bring about change and improve practice and clinical effectiveness.
The key messages being that:

1. Clinical audit is not just a data collection exercise:
./ It involves measuring current patient care and outcomes against explicit audit
criteria (also termed standards) .
./ There is.an expectation from the outset that practice will be improved.
2. Further dinical audit may be required to confirm that practice has improved.

Clinical audit should:
o Be a multi-disciplinary, multi-professional activity.

o Follow general accepted rules and standards which are based on international,
national or local legal regulations, or on guidelines developed by international,
national or local medical and clinical professional societies.

o
o
Be a systematic and continuing activity. whereby the recommendations
audit reports are implemented.
NOT be research, quality system audit, accreditation or regulatory activity.
given in


'.
t.

The general objectives of clinical audit should be to:
a Improve the quality of patient care.
a Promote the effective use of resources .
'.
a Enhance the provision and organization of clinical services .
a Further professional education and training.
Clinical audit should:
, ."
a
a
Address the practical clinical work by different
Assess the
consideration
local practice
the local facilities
against the
professionals .
defined good
and resources when the ultimate
practice, taking
good practice
into

a
cannot be reached by one step .
Have professional
professional
optimise
initiation
relationships
patient care.
and
and
the
foster an environment
multidisciplinary
which.
approach
enhances
required to
All parties, those being audited

confidentiality
and other performance
and those carrying
of patient data, the interviews
data .
out the audit, should respect
and discussions with staff, audit reports
the
'.
The History of Clinical Audit
Monitoring of patient outcomes had been carried out as early as 1750 BC, during king
Hammurabi of Babylon, with punishments for clinicians for poor performance. However, in

the modern
audit.
standards
hospital
era Florence Nightingale
She carried
following
patients.
this out during
an "audit"
the
is regarded as one of the earliest pioneers of clinical
on cleanliness
Crimean war of 1853"5, when
remarkably
Ernest Codman is known as the first true medical auditor
improvement
reduced the mortality rates in
following
of
his
work on monitoring
remained
care happened
surgical outcomes in 1912. Since then process of evaluating
static for many years. First move to integrate
in United Kingdom. Medical audit was introduced
clinical audit in professional
patient care
in the 1989 White Paper

health
'Working for Patients' which stated that systematic peer review of medical care should be
part of the routine clinical practice of all doctors. The process which was initially
as a medical audit subsequently evolved into a clinical audit .
established
Clinical audit is gaining popularity

strategies and as part of accreditation
in health services as a first step in quality
processes .
improvement
.'.
Using the method
Clinical audit can be described as a cycle or a spiral (see figure 1). Within the cycle there are
~.
stages that follow a systematic process of establishing best practice, measuring care against
'. criteria, taking action to improve care, and monitoring to sustain improvement. The spiral
suggests that as the process continues, each cycle aspires to a higher level of quality .

Clinical audit requires the use of a broad range of methods from a number of disciplines, for
example, organizational development, statistics, and information management. Clinical
audit can be undertaken by individual healthcare staff, or groups of professionals in single or
multidisciplinary teams. At the opposite end of the scale, a clinical audit project may involve
all services in a region or even in the country. Effective systems for managing the audit
project and implementing change are important whether a large number of people or only a

J

few are involved in the audit project. At the start of an audit project, spending time on
creating the right environment may be more important than spending time on the method
itself.
.
to
'
r--- -----,
( ,sa~plin;,'
- --
----------~ { ...
Q:e5tjOn~a~~e",
I
-----r-----..J
Are we
achieving it?
, ... Dat~ .
I
I

... - design
- -'
' , \. collection '
... ~. ~

r ""; :': " '\~~,~

i ~ . ',., '
., ."" I
l .. _.- ---_.
"--'.' -
. -'

Fig. 1: The clinical audit cycle (adopted from NICEguidelines)

Stages
The main stages of the clinical audit process are:
1. Selecting a topic.

2. Agreeing standards of best practice (audit criteria).
3. Collecting data.
4. Analysing data against standards.

Interactive workshop on clinicalaudit Page 8 of 117

.
'
---------------------------------------------------------
5.
6.
Feeding back results .
Discussing possible changes.
7.
8.
9.
Implementing agreed changes,
Allowing time for changes to embed before re-auditing,
Collecting a second set of data .
10. Analysing the re-audit data.

11. Feeding back the re-audit results .
12, Discussing whether practice has improved .

This process is summarised in the diagram below (figure 2) .
.' Compare
dalawith
standards
~)

Fig. 2: The clinical audit process
What Clinical Audit Is Not?
Not all 'audit' that takes place within the health service is clinical audit. Clinical audit is a
specific activity that measures clinical care against explicit audit criteria (standards) as part
of a quality improvement cycle. The term 'audit' has a range of meanings and whilst people
might want to 'audit' something it does not necessarily mean that they are doing or want to
do a clinical audit project.
Other forms of audit can include:
Financial audit - Looking at accounts to establish whether they provide a true and fair
view of the organization's financial position at a given time .

>- Internal audit - An internal mechanism that traces non-clinical activities and systems
along 'audit paths' to see if things happened the way they should have. For example,

tracing a patient complaint from the initial letter of complaint through to resolution to
establish whether Trust guidelines were followed appropriately.
~ Organizational audit - An external, independent and voluntary audit of the whole

organization, based on a framework of explicit standards. Organizational audit looks at
how well the organi,ation is set up and runs on a daily basis.
-, Counting things/ Investigations - The collection of data which is not related to explicit

audit criteria (standards) is not considered to be clinical audit.
-, Routine monitoring of clinical outcomes - The identification and measurement of
clinical outcomes that are explicitly linked to the change process may form part of a
clinical audit project
performance
However routine
monitoring is not considered
monitoring of outcome
being a clinical audit.
-, Peer review including Mortality & Morbidity (M&M) - Peer review is a process whereby
data for purposes such as

a group of clinicians collectively
care to establish whether
have been done differently.
at specific, non-random,
M&M
assess a small sample of patients
the best possible care was provided
reporting
recently
or whether
under their
things might
is a specific peer review process that looks
cases with adverse outcomes, such as death or injury, to see

what lessons can be drawn.
~ Staff, patient, service user, carer surveys - Surveys are usually carried out as part of a
research project or as an engagement activity. They are primarily
opinions of staff, patients, service users or carers regarding treatment
of care in order to see if improvements
used to gain the
and/or the quality
can be made. Surveys should only be used for

..'
clinical audit if the data sought cannot be collected from another source and it is related
to processes or outcome of care i.e. were standards of best practice being met.
Clinical Audit and Research: What Is The Difference?
"Research is concerned with discovering the right thing to do; audit with ensuring that it is

done right" tit
Smith R. Audit & Research. BMJ 1992; 305: 905-6
Research addresses clearly defined questions and hypotheses using systematic processes to

generate new evidence to refute, support or develop a hypothesis, by asking the question

'what
developed.
is best practice?'
The methodology
As a result of which a new service
is designed so that it can be replicated
can be generalised to other similar groups.
or new practice may be
and so that the results

Research may involve a completely
placebo treatment for purposes
new treatment
of comparison,
or practice, the use of control
or allocating service users randomly
groups or
to

different treatment
analysis of the work.
groups. Patients should be involved in the design, implementation and


'.
~
4)
Alternatively,
outcomes
implementing
clinical audit aims to improve
through
change
the peer-led
where
review
necessary,
the quality
of practice
It asks the
of local p3tient
against
questions
evidence-based
care and clinical
'are we following
standards,
best
practice?' and 'what is happening to patients as a result?'
~
Clinical
providers,
audit is initiated
including
by national
loc31 healthcare
address clearly defined audit questions
bodies, various healthcare
staff and managers. The methodology
that establish whether
stakeholders or service
is designed to
a specific clinical standard is
being met. Results are specific and local to a particular

tool may be used by more than one team or service .
team or service although the audit
A clinical audit project will never involve a completely

involve
patients
the use of control
randomly to different
groups or placebo
treatment groups.
treatments,
new treatment or practice,
nor does it involve
It may, however, Involve
never
allocating
input from
" patients, service users or carers at a number of levels, e.g .
Participation in surveys which help to determine whether standards have been met.
Involvement in the design of individual clinical aUdii~TOjects or whole programmes
./
of activity .
Research Audit
I May involve experiments based on a Never involves experiments and involves
. '
hypothesis .
It is a systematic investigation .
It may involve random allocation .

measuring against pre-existing
It is a systematic review of practice.

standards.

It never involves random allocation.
There may be extra disturbance to patients . There is little disturbance to patients.
It could be a new treatment. It never involves a completely new

treatment .
Creates new knowledge about effectiveness Answers the question "are we following best
of treatment approaches

~ practice ?"
May involve experiments on patients . Patients continue to experience their normal

treatment management.
It is usually a lengthy
large numbers of patients .
process and involves It is usually
number
carried
of patients
out involving
and within
a small
a short time
span .
'. Interactive workshop on clinical audit Page 11 of 117

It is based on a scientifically valid sample size It is more likely to be conducted on a ~
(except in the case of some pilot studies). pragmatically based sample size.
Extensive statistical analysis of data is Some statistics may be useful. '.

routine. Data analysis can take a number of
i
forms depending on whether qualitative or
quantitative research has been carried out.

t,
Results can be generalizable and hence
publishable. Quantitative research tends to
be more easily generalizable than qualitative
Results are only relevant within
practice settings (although the audit process
may be of interest to a wider audience and
local

work.
Responsibility to act on findings is unclear.

hence audits are publishable).
Responsibility to act on findings rests with

individual clinicians.

/
~
Findings influence the activities of clinical
practice as a whole.
Findings influence activities of practitioners
within a practice.

.~;/
Always requires ethical approval.
Research can identify areas for audit.

Does not require ethical approval.
Audit can be a precursor to clinical research

=
by pinpointing where research evidence is

lacking.

Table 1: Difference between audit and research
Adapted from "A Brief Introduction to Clinical Audit" by Carol Fawkes ".-
Clinical audit and research: Why it is important to know the difference?
As outlined above research projects and clinical audit projects have very different purposes, ,t
and therefore use different methodologies; they are also managed and funded in different
ways.
It is sometimes suggested that research is more rigorous than audit but research and audit

.,
can both either be rigorous Le. done according to protocol and producing valid results or -'
not rigorous enough Le. done carelessly, producing flawed results, and in the case of clinical
audit, not leading to improvements in clinical practice.
Whilst research requires ethics committee approval, clinical audit does not. However,
clinical audit should still be conducted within an ethical framework. Whilst clinical audit
projects may be published without ethical approval, journal editors may refuse to publish

articles if there are ethical concerns and ethics committee approval has not been granted. If

Interactive workshop on clinicalaudit Page12 of 117

the organization wants to publish because of the results of the project, rather than to share
the methodology, then the organization
research, rather than a clinical audit project.
should question whether they are undertaking
Difference between Service Evaluation and Clinical Audit
., The aim of service evaluation is to judge a service's effectiveness o~ efficiency through the
systematic assessment of its aims, objectives, activities, outputs, outcomes and costs. It
addresses specific questions about the service concerned and results are specific and local
to a particular team or service and may lead to service redesign and reconfiguration in that
particular
service .
area. The evaluation tool may, however, be used by more than one team or
Service evaluation never involves completely

groups or a placebo treatment
new treatment
nor does it involve allocating
practices; the use of control
service users randomly to
different treatment groups. It may, however, involve input from patients, service users or
carers through their participation in surveys, which help to determine the effectiveness or
efficiency of a service, or through their involvement in the design of individual projects or
t
whole programmes of activity .
Patient, Service User, Carer Engagement
Research, clinical audit Dr service evaluation projects may all include a patient, service user
or carer survey .
In terms of clinical audit, surveys can be a useful tool, where measuring compliance against
your audit criteria requires information that can only be obtained from the patient or
.- service
appointment?'
user e.g. 'Did the doctor introduce themselves
'Did the doctor listen to what you had to say?'
at the beginning of your
Surveys can be construed as doing something 'beyond normal clinical

therefore it is important to get advice on the design of your survey as some questions might
management';
touch upon potentially sensitive matters, which would give rise to ethical concerns. It is
extremely important that all surveys are designed to cause minimum possible disruption .
Is Clinical Audit and Medical Audit Synonymous?
'Clinical audit' tends to be used as an umbrella term for any audit conducted by
professionals in health care. Audits conducted by doctors are often referred to as medical
audits, although the term 'clinical audits' could also be used. It is important
very few health care procedures involve just one professional discipline and that non-clinical
to stress that
staff such as receptionists,

quality of the service provided.
secretaries, nurses, pharmacists,
Clinical audit, therefore,
etc. playa vital role in the
is usually a multi-disciplinary
Interactive workshop on clinical audit Page13 of 117

activity. Many clinical audits are also 'multi-sectoral',
social services, primary and acute care providers,
that is, they may involve
education and health.
health and

Summary '-'I
Clinical Audit is a quality Improvement process that measures current patient care and

outcomes against agreed standards of best practice.
Not all 'audit' is clinical audit.
Be aware of the differences between project categories:

o Clinical audit - audit against agreed standards of best practice.
o Research - aims to create new knowledge.
o Service Evaluation - assesses the effectiveness of a service.
Acknowledgements
1. A Brief Introduction to Clinical Audit; Carol Fawkes

2. Principles for Best Practice in Clinical Audit: NICE guidelines published in 2002
3.
4.
What is clinical audit? (Version 3); Published by UH Bristol (University
What is clinical audit?; Tracey Jones, Simon Cawthorn
Hospitals, Bristol)

.~
...

'


i.
The NHSexecutive's 'original'
definition
"The systematic critical analysis of the quality of health

care, including the procedures used for diagnosis,

treatment and care, the use of resources and the
resulting outcome and cuality of life for patients. It
embraces the work of all healthcare ~rofessionals"
The DOH. Working tor PGtieflts, Londo!): The Stationery Office, 1989

'.
Definition (NICE)
"A quality improvement process that seeks
to improve patient care and outcomes
through systematic review of care

against explicit criteria and the
implementation of change."

Definition (NICE)
Aspects of the structure~ processes~ and
outcomes of care are selected and

systematically
criteria .
evaluated
Where indicated, changes are implemented at

against explicit
an individual, tean, or service level and

further monitoring is used to confirm
improvement in hea thcare delivery .

Interactive workshop on clinical audit Page IS of 117

Clinical Audit ... .,
An important tools for carrying out QA
activities.
Is the systematic analysis of the quality of

healthcare, including
- the procedures used -for diagnosis,
- treatment and care,
- the use of resources cmd

- the resulting outcome and quality of life for the
patient.

Clinical Audit-Types

Standards based audits
- To see if standards are met or are being improved
Adverse / critical incident

- Screening of such inciden:s
- Monitoring
Peer review
of such incidents

- Was the c:uality of care optimaL
discussions ':often mutt[di~ciplinary)
Patient surveys
Case reviews
- Patients poi'lt of view of the quality of service

and

Clinical Audit-Process

5. Implement change 2. Set criteria
and standards

~
4. Compare performance 3. Observe practicel

withcriteria andstandardSffJ data collection


What are the Pre-requisites?
Good record keeping system

Should be carried
professionals
out by fair and impartial

~'
Clinicians, nursing a1d other staff as well as
patient anonymity to Je maintained
Initiative should come from within
Purpose should be simple and clearly stated
- Intention should be to effect change for the

better
Similarities between clinical audit
and research
both aim to answer a specific question

relating to quality of care
both can be carried out either retrospectively
(looking at historical data) or prospectively

(collecting data as care is given)
both involve careful sampling, questionnaire
design and analysis of findings

The interface between Clinical
Audit and Research...
Clinical audit can be legitimately viewed as the

final stage of a good clinical research
programme
Alternatively research could be viewed as a
precursor to the clinical audit process

Research can identif'{ areas for audit

.1

The interface between Clinical
Audit and Research

Audit can pinpoint areas where the research
evidence is lacking

The audit process assists with
dissemination of evidence-based practice
the

IMPORTANCE

Professional accountability I
Changing attitudes
- Questioning
- Demands of pressure groups, press coverage, calis
for public inquiries, and

- Rise of complaints,
redress.
Yet all is not lost

legal challenges and claims for

Retain the trust and respect in an increasingly
critical environment.


Benefits: Health Care
Provides
Professionals
workable
Resolves problems
standards
Improves and increases

levels of communication
Ensures appropriate
team-working and
use of skills and resources

...~
Increases knowledge and skills
Can identify training needs
Measures quality in current practice
Why should I bother doing clinical

audit?
Critical analysis of your own skills and ways of

working (self improvement)
Critical look at the system that delivers care to

your patients (i.e. your environment)
Can make a real difference and change things

for the better

,. Benefits: Patients
Improves quality of care and service received
Prompt changes in de'ivery

Highlights precise patient needs
Involves patients in decision-making
of care
Raises patients confidence in service and care

ievels
Provides clear
risks involved
information about care and

Benefits: Organization
Improved care of patients

Enhanced professionalism
Efficient use of resources
Aids in continuing education

of staff

Aids in administration
Accountability to those outside the profession

-,~
Overall Benefits

Best practice
Best outcome

Best that
collectively
we can deliver
as a group of health professionals
individually or

~""I<:I"",,'!\,,",lWO'.~lop


NOTES
-"C. ~
~'~-~"~.o,,,-
.<. _10
.~=,
~
~
.; .
~

NOTES

-.
,


~/ .

Stage One: Preparing for Audit and Selection of Topic
Good
preparation
preparation is crucial
must be addressed:
to the success of an audit project. Two broad areas of
1.
2,
project
communication
project
management,
methodology,
including
including
topic selection, planning
design, data issues, implementability,

and resources,
stakeholder
and
involvement, and the provision of support for local improvement.
Providing a structure
To enhance the benefits of audit, an organization
a structured audit programme (committee

needs:
structure, feedback mechanisms, regular

audit meetings)
a team of well-qualified audit staff
Clinical audit projects are generally described as being either uni-disciplinary, e,g. involving
only
profession,
one staff group,
It is important
or multidisciplinary, e,g. involving more than
that a clinical audit project assesses patient care as provided
one discipline or
by

the whole clinical team, in order to identify how care can be improved, Therefore if the
project has implications for more than one profession or disciplines it is important to ensure
that they are represented on the project team,
Consider including a patient, service user and/ or carer representative(s)

or gaining their views on what they would like to see from the service, Engagement
in the project team
is

important as healthcarE is a partnership between clinicians and their patients/ service users,
Whilst we strive to provide the best quality of care as we see it, patients/ service users
might want something different. Direct engagement is best e,g, through participation of

identified individuals on project steering groups or divisional/specialty audit committees,
However, indirect engagement is another possibility e.g, through the completion of a
survey, usually administered at the end of an episode of care,
It is important
commitment
that the project is supported
to see that any necessary changes indicated
by colleagues who have the authority
by the audit results are put into
and

practice,
Identifying and Developing Skills for Audit Projects
To be successful, a clinical audit project needs to involve the right people with the right skills
from the outset, Therefore,

should be priorities .
identifying the skills required and organising the key individuals

.

Certain skills are needed for all audit projects, and these include:
,..
'
./ project leadership, project organization, project management
./ clinical, managerial, and other service input and leadership
./ audit method expertise

./ change management skills
./ data collection and data analysis skills
./
Audit teams
facilitation skills.
,
It is necessary to set up an audit team customised
providing
staff are usually included in audit teams.
to the specific audit, with team members
many of the skills needed. For example, clinical service representatives
It is also important that the team
and audit
includes
members from all the relevant
clinical experience.
processes of clinical
essential real-world
groups involved
care and the information

knowledge
in care delivery,
All audits need direct access to people with
systems used within
is most likely to be found from
and not just those with
a full understanding of the
the service, and this
the staff working in the
service.
All team members should have:

./ a basic understanding of clinical audit

./ an understanding of and commitment to the plans and objectives of the audit
./ an understanding of what is expected of the audit team - this needs to be clarified at
the outset and may be expressed in a 'terms of reference' document.
Finally, if the audit team is to improve the performance of a clinical service, team members I
must be able to communicate effectively with their colleagues. Members of the audit team

must, therefore,
able to promote
have the full confidence and support
the audit and plans for quality improvement.
of the staff and organization and be

Those responsible for clinical audits should:

Ensure a summary
recommendations
Ensure a summary
of clinical audit
for action, is distributed
of any recommendations
results, in an appropriate
to all relevant stakeholders.
format
and action plans is shared with other

and with

interested
Ensure that
identification
parties.
clinical audit results having
of a risk, are brought to the attention
potential significance,
of the management.
due to the

Present clinical audit results at various forums as appropriate.


Responsibilities of staff:
o
To undertake /cooperate in/register
Ensure they have training for their role .
Follow protocols on confidentiality
clinical audit.
and security of data
o Act on results .
Ethics and Engagement
Unlike research, clinical audit projects do not need to be submitted to an Ethics Committee
for ethical approval. This is one of the key reasons why it must be ensured that the project is
clinical audit rather than research .
Clinical audit should always be conducted within an ethical framework, This means abiding
by the principles of data protection e.g. by ensuring patient and staff confidentiality
ensuring that data is collected and stored appropriately .
and by
Selection of Topic
The starting point for many quality improvement initiatives - selecting a topic for audit -
needs careful thought

investment of resources .
and planning, because any clinical audit prcject needs a significant
Audit priorities
To choose an appropriate
helpful:
topic for a clinical audit project, the following activities may be

a.
b.
At an audit team
perceived importance .
Consult with
meeting,
any other
discuss
relevant
possible
stakeholders
topics
(not
and prioritise
on the audit
according
team)
to
about
c.
proposed topics .
Evaluate the topics according to the criteria outlined
The clinical team has an important role in prioritising

below .
clinical topics, and the following

questions may be a useful discussion guide .
./ Is the topic concerned of high cost, volume, or risk to staff or users?
./
./
Is there evidence of a serious quality
high complication
Is the topic financially
rates?
important
problem, for example patient
(either very common and/or very expensive)?

complaints or
,( Is good evidence available to inform

national clinical guidelines?
standards, for example systematic reviews or

,( Is the problem concerned amenable to change?
Interactive workshop on clinical audit Page 2S of 117

,/
,/
Is there potential for involvement in a national audit project?
Is the topic pertinent to national policy initiatives?

,/ Is the topic a priority for the organization?
What can be audited?

The quality
component parts:
of health care provided can be audited by examining three interrelated

Structure
Process

Outcome
Audits of structure
This type of audit looks at environmental

can be considered
confidentiality
receptionist,
include
offered
clinicians,
during
other
the
factors within which care is delivered. Criteria that
practice
consultations,
health
building
privacy,
care practitioners
(state of repair,
cleanliness),
and additional
the
facilities
personnel
ancillary
offered,
(the
staff),

equipment in the practice (is it always functioning, is it regularly assessed for safety) and
patient
complete,
notes (are they kept securely to maintain confidentiality,
are they of a suitably high standard). This provides an indirect assessment of a
are they legible and

patient's care, but the environment in which a patient is treated is, nonetheless, an
important aspect of their care.
Audits of process

Audits
treatments,
of process focus on the clinical care received by patients
or procedures. Projects are best focussed on the processes, which have been
shown result in the best patient outcomes. For example if research has shown that Drug X
gives better outcomes than Drug Y for patients with condition
e.g. investigations,
A, you would audit "are

patients with condition A being given Drug X?" This type of audit can focus on the technical

skills of a clinician and an evaluation of the decisions made concerning the management of a
patient.
Audits of outcome
Outcomes are considered to be the most relevant

examine the change in the health status of a patient
assessment of a patient's
following a particular
care. They
treatment

intervention. An extensive number of outcome measurements
assess general health status, physical health and psychological wellbeing. Outcome audits
have been developed to

can be concerned with:
Response to treatment in terms of pain relief or change in levels of disability
' .

Response to treatment in terms of reaction to treatment e.g. soreness, increased
pain or disability within a specified time frame.

Degree by which patients can manage their symptoms following advice delivered .
Disadvantages of auditing structure and outcome
Audits of structure
staffing numbers
look at the resources that enable treatment
and mix, as well as, environment and equipment.
and care to happen; such as
Clinical audit is not
.' usually used to audit structure. Structural improvements usually involve

better dealt with via business plans or, perhaps, other forms of audit such as financial audit,
internal audit or organi2ational audit .
money and are
Audits of outcome
in health
complication
status,
look at the results of interventions.
examples
rates, readmission
include dead/alive
Outcomes
(mortality),
are measureable
recurrence
rates and quality of life measures both generic and disease
of
changes
disease,
specific. Measuring outcomes can be difficult. Clinical outcomes are often routinely
addressed through
Defining the Purpose

peer review processes including Mortality & Morbidity (M&M) reporting .
A project without
be established
clear objectives
before appropriate
cannot achieve anything: a clear sense of purpose
methods for audit can be considered. Once the topic for a
must
clinical
defined,
audit project
audit project has been selected,
so that a suitable audit method
therefore, the purpose of the project
can be chosen. To define the aim of your clinical
consider what it is that you hope to achieve i.e. the overall purpose
must be
of the
project. It should be related to the rationale

not merely be to 'count the number
achieving improvements
behind choosing your audit topic and should
of' or 'examine'
in practice where necessary .
but should focus your audit towards
The aim can be written

audit. Statements should
as a statement
be phrased
about what you want to happen as a result of the
positively, to ensure that the audit brings about
improvements
of an audit (Buttery,
in practice. The following
1998):
series of verbs may be useful in defining the aims
./" to improve
./ to enhance
./ to ensure
./ to change .
Examples of using these are:
./
./
to improve the blood transfusion
to increase the proportion
controlled
processes within the organization
of patients with hypertension whose blood pressure is
Interactive workshop on clinical audit
Page 27 of 117
." I
.1

if to ensure that every infant has access to immunisation
pertussis and polio before 6 months of age.
against diphtheria, tetanus,

The aim can also be phrased as a question that you want your audit to answer. For example,

'Are we meeting standards of best practice for the management
Objectives
The aim provides a broad structure
of leg ulcers?'
for the clinical audit project. This may need to be broken

. .'
down into a series of smaller objectives. Objectives are the steps that need to be taken in

order to assess whether or not the aim has been achieved. The objectives can be written
either specific tasks to be undertaken or as the different aspects of quality that the project
as

will address.
A simple and illuminating question that could be asked by the team when designing the
project is: "What do we think that we ought to be doing?" The answers to this question
should be based upon the best available evidence.
More technical forms ofthis question are outlined below,

Aspects of Quality
The majority of clinical audit projects focus on:

o Appropriateness - Is the right treatment being provided to the right patient? It is
generally
appropriate
easier to assess whether
rather than whether
the patients
all the appropriate
that received the
patients received the treatment.
treatment were

o Timeliness - Was the treatment
linked to treatment.
patient within
Other
given at the right time? Timeliness
examples
15 minutes, an appointment
referral to treatment
include the
for fast-track
within 18 weeks, arrival to bed allocation
medical review
is not exclusively
of a deteriorating
cancer referral within
in ,0, & E within 4 hours.
14 days,
o Effectiveness - Was the treatment given in the right way? With desired effect?
Other aspects of quality that do not tend to be assessed through clinical audit are:

o Efficiency - Was the treatment given with minimum effort, expense, waste? Efficiency
issues are best resolved through service improvement work. Service improvement aims
o
to improve processes and systems of care, through
order to make them more efficient.
Acceptability - Is the treatment
process mapping
acceptable to the patient? Acceptability

and redesign,
is usually a focus
in

o
o
of research or patient involvement
Accessibility - Ease of getting care.
Equity - Is the treatment
activity, rather than clinical audit.
available to all patients on an impartial basis?

,t

Accessibility
the structural
and equity issues are best dealt with by management
aspects of care .
addressing problems with

EXAMPLE 1 How to Write: Clinical audit topic, aim and objectives
Topic: Leg ulcers
Aim: To improve the care received by patients with leg ulcers
Objectives:
1. To ensure that leg ulcers are treated appropriately

2. To ensure timely treatment of leg ulcers
Patient care cannot be measured against the objectives

they do not specify what 'appropriate' treatment
contained
is, or what
in the example above, as
'timely' treatment would
consist of. This is where audit standards come in. Audit standards define exactly how these
aspects of care will be measured .
Summary
Focus your efforts where there is greatest potential for improving the quality of care. Do
not waste valuable time

possibility
looking
of making improvements
at areas where
.
realistically you know there is little

Get all your stakeholders, colleagues, managers, etc., on board from the start and make
sure that they understand clearly what you are trying to achieve.
I Clinical audit needs to be justifiable in terms of the benefits it will bring about for
patients balanced against the amount of time and resources it takes. For each proposed
project topic, ask yourself:
o What is the benefit for the patient of doing this project?
Acknowledgements
Will it take a disproportionate amount of time and/or funds to complete?
1. A Brief Introduction
2. Introduction
Bristol)
to Clinical Audit; Carol Fawkes
to clinical audit (Version 3); Published by UH Bristol (University Hospitals,

3.
4.
Principles for Best Practice in Clinical Audit: NICE guidelines
How To: Choose & Prioritise
Hospitals, Bristol)
Topics (Version 3); Published
published in 2002
by UH Bristol (University
.
5. How To: Set an Audit Aim, Objectives & Standards (Version 3); Published by UH Bristol
(University Hospitals, Bristol)
'
Interactive workshop on c1ioicalaudit

Page 29 of 117
.1

Five elements
,
1. Involving users in the process
2. Topic selection
3. Defining the purpose of the audit

4. Providing the necessarystructures

5. Identifying the skills and people needed to
carry out the audit, and training staff and
encouraging them to participate.

Involving users

The focus of any audit project must be those
receiving care.

Users can be genuine collaborators, rather
than merely sources of data.
8r;/ogh ~I "t.. 1995

Sources of information

Letters containing comments or complaints
Critical incident reports

Individual patients' stories or feedback from
focus groups
Direct observation of care

Direct conversations
Balogh e't D~, 1995


Involving users: Commission of
Health Improvement (CHI) Report
Integral part of the quality improvement and

clinical governance strategies
' .
Satisfaction survey
Clinical
incidents,
CHI Experience
audits In
complaints,
response
NICE
to
guidance
National Service Frameworks were seldom
reported
or
performed
Few multidisciplinary audits were undertaken

Patients'
considered
perspectives were not generally

There was no systematic implementation or
follow-up of audit findings
Providing a structure
A structured audit programme
- committee structure,
- feedback mechanisms,
- regular audit meeting~
A team of well-qualified audit staff
DJ,kimo~ 8<Edwards ]999

.

Structure
Resources required

- numbers of staff and skill mix
- organizational arrangements
- provision of equipment

- physical space.

Process '.

Actions and decisions. taken by practitioners
together with users _-;-._
- communication
- assessment

- education
- investigations
- prescribing
- surgical and other therapeutic interventions
- evaluation
- documentation

Outcome
Measures of the
response
- interventicn
physical or behavioural

- reported health status
-level of knowledge
- satisfaction


'1

Which to choose?
Clinical Audits are best focussed around processes

Clinical Audit usually focuses on measuring whether

we are following best practice processes, i.e, that
which will bring about the best outcome. However:
"If you measure outcome, you won't know how

you got there and if you measure process you
might nat know what happened at the end of it"

Charles Shaw (1998)

Some topics
long/ short stay cases
Specific diseasej specific operations

Vulnerable groups
Increase incidence of a disease

Post operative infection/ complications

"('4"' l.,",,-
o
"" $.vd'.lil,
o

{'-""'~"'-

Page 33 of 117
~ C)-<1'C
.'

'
Exercise 1
Kindly identify if the below mentioned parameter corresponds to structure,

process or outcome.

51.No.
1.
Parameter
Patient death or serious disability

Structure Process Outcome

associated with the use of contaminated
drugs, devices, malfunction of devices, or
biologics pro~ided by the healthcare facility; /'

2.
accidents, incidents
Anti-thrombotic therapy - Percentage of
patients with an INR above 4 whose dosage

3.
has been adjusted or reviewed prior to the
next warfarin dose
Patients satisfied with their current pain
,/
,j

relief.
4. Medication
patients
ordering
whose known
- Percentage
adverse
reactions are documented on the current
of
drug
v'
s.
medication chart
Aspirin for AMI at discharge .;
6. Availability and use of information
electronic clinical information; management
information; staff development
-
v'

information, quality reports

7. Medication ordering - Percentage
paediatric medication orders that include
the correct dose per kilogram (or body
of
j

surface area) and a safe total dose

8.
9.
Initial
guidelines
antibiotic consistent
Foreign body left during procedure

with current
,i
\ ..

10. Postoperative pulmonary embolus or deep
vein thrombosis
v
11. The number of specialists that contributed
to complicatiJns registration system
,/


SI. No. Parameter Structure Process Outcome

12. Prophylactic antibiotic administration
(surgical patients) v
13. Coronary artery bypass graft (CABG) using
.
internal mammary artery
14. Tonsillectomy
haemorrhage
- significant reactionary
v
15. Aspirin at arrival for AMI

v


It

. --
.
-
~-
Exercise 2
-. 1

Identify a topic for clinical audit in your group.
Kindly write the purpose and objectives for the audit.
~
-"
,f

.~."
~
-'

'-"
"'- '
. .:.
-
-.

.

, ~

""

.",
,

I
i
~
rI
. NOTES
'
''..:=<
-
.',
,

"-
,~
,
I
.
,,-
. ~':
.-
'
.
" ,
'.
'. ~

.'

NOTES


' .

Stage Two: Selecting Criteria
{Adapted from - How To: Set an Audit Aim, Objectives & Standards (Version
3); Published by UH Bristol (University Hospitals, Bristol)}
Audit Standards
Clinical audit is by definition standards-based (sometimes referred to as 'criterion-based') .
Standards are more specific than objectives.

specific aspects of patient care and/or
They are quantifiable
management
statements
that the audit intends
detailing
to measure
the

current practice against. They seek to ensure that the best possible care is provided, given
available resources, and they are based upon the best available evidence .
"A standard is an explicit statement describing the quality af care ta be achieved, which is
definable and measurable"
The standards
below .
should be related to the audit topic and objectives, as per the framework
Audit'
topic .
Aim

Objective Objective
......
I. StandardS
...
Standard Standard Standard
. .
The Importance of Setting Standards

Fig. 3: Framework
Standards
facilitates
play an important
discussion among
role in the
staff about
clinical
a particular
audit process. Developing
aspect of care and inspires
standards
some

reading of the relevant literature. Comparing current practice against standards can
highlight problems which may otherwise have remained unrecognised. Standards may help

.

to motivate changes in practice
and the desired level of care provision.
by revealing gaps between the quality of current practice

How to Set Standards

If standards are available in the form of guidelines or protocols, the audit should be based
on the most widely applicable guidelines available, e.g. national guidelines rather than
regional or local guidelines. However, it is important to bear in mind that guidelines are only
as good as the evidence they are based on. Even some national
only 'good practice', without any research evidence base.
guideline statements are

If guidelines/ protocols do not exist, or existing ones are out of date, a literature search to
identify best practice would have to be undertaken.
It is important that there is agreement with the standards internally before the audit starts.
The organization
constitutes
will find it hard to improve
best practice.
practice if there is disagreement as to what

Standards
constitutes
should alwa\'s be based on the strongest,
best practice. A generally
base your standards on is:
accepted hierarchy
most up-to-date,
for the strength
evidence of what
of evidence to

Stronger (best) Systematic Reviews of RCTs (randomised controlled triol-
eliminates/reduces bios to give most reliable evidence)

Results of single RCTs

Results of well-conducted non-RCT clinical studies

Expert committee
authorities
reports; clinical experience of respected

Weaker Personal experience and opinion
Where Standards Come From
Standards may be based on one, or any combination,
National guidance or standards.

of the following:

College or professional
Laws.
organization
Previously agreed local guidelines/protocols.

guidelines.

Standards used locally by colleagues or competitors.
Research evidence (from which standards can be developed).


Literature review of other clinical audits which have published their
standa rds/results.
Current knowledge from clinical experience .
Current practice (observe and assess current practice) .
Understanding Standards
All standard statements
Standard
should contain a criterion
= criterion
and a target, as shown below .
+ target
(Statement of what
is being measured) + (Yardstick)
~

(%to be achieved)

(measurement boundary)
Audit Criteria - The audit criteria quantifies the practice addressed by the objective,
describing in a measurable way what care should be delivered. The audit criteria should be
derived from the evidence-base. The stronger the evidence base, the more likely it is that
staff will agree with the audit standards and therefore
making changes if the audit shows the standards are not currently
they will be more likely to commit to
being met.
A criterion:
forms the main body of the standard

is a clear and precise statement
uses words/phrases
of care
which mean that it is measurable

indicates the bOJndaries of the measurement (e.g. a time frame and who it involves)
known as a yardstick .
A target:

is expressed as a percentage and defines the level of performance considered
acceptable, in relation to the chosen criterion .
Below is an example of a standard statement about response times which contains all of the
necessary components .
Target (Criterion - in brackets)
t
95% of (children referred to the department
two weeks of the referral being received)
will be seen by a member
~
t
of the team within
Yardstick-in italic


Setting Targets
Targets should be set at realistic

setting targets the following
and attainable levels, while not being set too low. When
factors should be considered:

clinical importance
practicability
acceptability
In the above example the target is set at 9S%. A target of 100% may be unrealistic at times.

Exceptions - The exceptions are the justifiable reasons for not providing the level of care
specified. For example it is not a failure on the part of the organization if a patient chooses
not to accept care. A common exception is patient choice, for example, if a patient was not
seen within a standard of 4 weeks from referral because they were away on holiday.
Consensus on the list of exceptions

careful that an exception
should be achieved before
isn't a failure in care in disguise e.g: 'patient
the start of an audit.
choice' may mask the
Be

fact that
confidently
the patient was not given sufficient
agree to treatment.
information about risks and benefits to

Sometimes
circumstances
it may be possible, prior to the
when it would acceptable for a criterion
be more sensible to set a target of 100% with
clinical
defined
audit being conducted, to identify
not to be met. In this situation
exceptions. An example
it may
is shown

below.
For 100% of adolescents attending the therapy group, a letter will be sent to their GP

prior to attending their first group session explaining why the adolescent has been
asked to attend and over what time period.

Exceptions Cases when consent to contact the GP is denied by the client.
Developing Good Standards
When writing your standards try to remember
y . Specific - Clear, unambiguous

that they should always be
and jargon-free. A standard

SMART:
should only mean one

'y
thing to all people who read them.
Measurable
example,
- Is the information
"information leaflet
required
should
to answer the standard
be given to patients".
available?
If data is collected
For
retrospectively,
of documentation.
it may not be possible to know if it's a failure of practice or a failure

'y Agreed - By all concerned with delivering that aspect of care.
'y Relevant - To area of care being audited I concern that has been raised.

>- Theoretically sound - Based on evidence about best practice, reviewed and updated
as new evidence becomes available .
EXAMPLE 2 How to Write: An audit standard
Topic: Leg ulcers

Aim: To improve the care received by patients with leg ulcers
Objectives:
1. To ensure that leg ulcers are treated appropriately
Audit standard:
1. 100% of venous leg ulcers will be treated

bandaging .
with graduated
Exceptions: ABPI (ankle: brachial pressure index) < 0.8

multi-layer high compression
How Are These Standards Used To Measure tare?
At the end of the data collection and analysis there will be three groups of patients:

i. Those who confDrmed tD the standards .
ii. Those who did not confDrm tD the standards but met the exception criteria.
iii. ThDse who neither conform to the standards nor met the exception criteria .
As a result of the data collection

for not meeting the stilndard,
the audit may discover that there are Dther valid reasons
which had not been previDusly considered as exceptiDns. It is

therefore important, for the audit team, to scrutinise the cases in the third category in order
to decide whether the reaSDns fDr them nDt meeting the audit criteria are acceptable Dr
whether they identify failures in care that can and shDuld be rectified .
Summary
InvDlveall"stakeholders from the outset.

Review the evidence base.
Set SMART standards .

Keep the project focussed. DD nDt set tOD many standards .
Acknowledgements
1. Undertaking
Psychiatrists
a clinical audit project: a step-by-step guide; Published by RDyal CDliege Df

.-

Please remember

There are other definitions for standard
and criterion.

Definitions of a 'criterion'
An item or veriable which enables the achievement
of a standard (objective of care) and evaluation of

whether it has been achieved or not.
4RoyaJ Co1!ege of Nursing, 1990
A definable and measurable item of healthcare
which describes quality and which can be used to

assess it.

-Irvine and Jrl.!in~, 1991
A systematically developed statement that can be

used to assess the appropriateness of specific
healthcare decisions, services and outcomes

-Insritvte of Medicine, 1992

Definitions of a 'standard'
An objective with guidance for its achievement given
in the form of criteria sets which specify required
resources, acti'lities, and predicted outcomes
~Roy!ll College of Nursing, 1990

The level of care to be achieved for any particular
criterion
/rlJine and Irvine, 1991

The percentage of events that should comply with
the criterion
-Baker and Fraser, 1995


Howto set?
Guidelines and protocols
Literature search
- should always be b"asedon the strongest, most up-

to-date, evidence of what constitutes best
practice
Developing good standards ...
Based on evidence
Related to important aspects of care
Measurable
Group work
Hc:am5fww c:r oj 2001

Developing good standards ..
To be set prior to the audit commencement
Criteria to be based on objective measures

- Explicit or implicit
Criteria should be well justified
Target should be set at realistic level for

defined patient groups and take into account
local circumstances


Using guidelines

Recommendations from clinical practice
guidelines can be used to develop criteria and

standards without substantial additional work.
-SIGN
- NICE

Other methods for developing

Prioritising the evidence method
RAND/UCLA appropriateness method
AHCPRmethod

Prioritising the evidence method
Identify key elements of care (review of good-
quality guidelines or systematic reviews)

Carry out focused systematic literature reviews in
relation to each key element of care to develop
Prioritise the criteria into 'must do' or 'shoufd do'
on the strength of research evidence and impact

on outcome.
Present the criteria in a protocol.
Include data collection forms, instructions etc.
Submit the protocol to external peer review??
ftMf'rl'ro/1997


Must Do criteria
New benzodiazepine preicriptions must only be issued

for short-term relief (no longer than four weeks) of
severe anxiety or insomnia

The records show that a oatient receiving a prescription
(either new or repeat) for a benzodiazeplne has been
advised on non-drug therap.ies for anxiety or insomnia
The records show that the patient has been given
appropriate advice on th2 risks, including the potential
for dependence
The records
benzodiazepines
monthly
show that patients
are reviewed regulcrly, at least three-
prescribed
51ww& Becby __Rpviewing benzajiolepine

prescriptions
".os~Cl.,.""A<.td't_st.op
Should Do criteria
The records show that, if the patient is aged 65 years

or over, they or their car.er(s}have been given advice

on the risks for elderly patients
Chronic users (use for 4 weeks or longer) should be
identified and encouraged to reduce
The drug taper should be gradual, with a reduction
of 2-2.Smg diazepam equivalent every 2 weeks

Before drug reduction is started, the patient has
been switched to an equivalent dose of diazepam

RAND/UClA appropriateness
9-point scale
method

- 1 (extremely inappropriate) to 9 (extremely appropr;ate)
Combines systematic review of the scientific literature with

expert opinion
Yields specific criteria that can be used for review criteria or

practice guidelines, or both
Provides a quantitative description of the expert judgement of

a multidisciplinary group of practitioners
Gives equal weight to each panelist in determining the final

rQsult
Kohn et "f 1.986

AHCPR method

Complex

Importance to quality of care
Feasibility for monitoring

AgenC)/ for Health Care Policy & Research 19950 & 1995b

Clinical Pathways

Synonyms
Integrated Care Pathways

Multidisciplinary
Pathways of Care
Care Maps
pathways of care

Critical pathways
Collaborative Care Pathways


._------------------------------------------- .

----_. __

What are they?
Structured, multi-disciplinary
to support the implementation
and protocols .
plans of care designed
of clinical guidelines
They are designed to support clinical management,

clinical and non-clinical
clinical audit and also financial management.
resource management,

They provide detailed guidance for each stage in the
management of a patient (treatments, interventions
etc.) with a specific condition over a given time
period, and include progress and outcomes details .
What are they?
They define the optimal sequencing and

timing of interventions by physicians, nurses,
and other staff for 3 particular diagnosis or

procedure.
It is a guide to plan, coordinate, deliver,

monitor, review
concurrently .
and document care

Definition
Multidisciplinary ma1agement tool based on

evidence-based practice for a specific group of
patients with a predictable clinical course, in
which the different tasks (interventions) by

the professionals involved in the patient care
are defined, optimized and sequenced either
by hour (ED), day (acute care) or visit

(homecare). Outcomes are tied to specific
interventions .

Aim

To improve, in particular, the continuity and

co-ordination of care across different
disciplines and sectors while minimizing costs.

Four main components

a timeline,
the categories of care or activities and their

interventions,
intermediate
and
and long term outcome criteria,

the variance record (:0 allow deviations to be
documented and anaysed).

H,II, 19!M, Hill J998

How do they differ from practice
guidelines?
They are utilised by a multidisciplinary team

and have a focus on the quality and co-
ordination of care

Interactive workshop on clinical audit Page so of 117

Benefits
Support the introduction of evidence-based

medicine and use of clinical guidelines;

Support clinical effectiveness, risk
management and clinical audit;

Improve multidisciplinary communication,
teamwork and care planning;
Can support continuity and co-ordination of
care across different clinical disciplines and

sectors;
Benefits
Provide explicit and well-defined standards for

care;
Help reduce variations in patient care (bV

promoting standardisation);
Help improve clinical outcomes;
Help improve and even reduce patient
documentation;
Support training;
Benefits
Optimise the management of resources;

Can help ensure qu,lity of care and provide a
means of continuous quality improvement;

Help manage clinical risk;
Help reduce costs bVshortening hospital stavs.


Involving users ...

Practitioners - Clinical competence and
measurable benefits to patient health status
or outcome.

Users, concerned that a holistic approach to
care is adopted and be more interested in
process criteria.

- different patient
perspectives
groups will have different

Involving users ...
Audit teams can collaborate with users to
establish their experience of the service and
the important elements of care from which
criteria can be develooed.

Several qualitative methods are available:
- ::ritical incident technique
- focus groups
- :onsumer audit

Involving users
Once the preferencEs of users have been
identified, they must be incorporated into the

criteria.
Best way of doing this?
- if the criteria selected by clinicians and those selected

by users relate to different elements of care, both sets
of criteria may be induced.
- If clinicians and users have different views about the
same element of care, an open approach is required
to achieve consensus.


Some useful resources
http://www.sign.ac. ukl
http://gu idanee.nice. J rg.u kl

http://www.guidelinE.gov/
http://www.medicinE.ox.a c.uk/ba nd01ier Ikn 0
wledge.html
Why?
-Information
Setting targets ...
about what other teams have
achieved
performing
can indicate
in relation
how
to others
well
the exchange of ideas about how practice can be

improved.
a
and encourage
unit is
Literature review: Did not find any evidence

Failure to reach a set performance level must
be examined carefully

Exceptions
Are the justifiable reasons for not providing

the level of care specified .
- For example it is not a failure on the pdrt of the
organization if a patient chooses not to accept
care .
Consensus on the list of exceptions should be

achieved before the start of an audit .


Exercise 3

, 1)N~J." J k .!-/e..._de-~
Off--< ,p:, Ic...

}-v
'kr~ s\ ~ ~~
..J.C--Uopic chosen by you in exercise 2.


i

NOTES


NOTES

Interactive workshop on clinical audit Page S6 of 117

Stage Three: Measuring Level of Performance
{Adapted
(Version
from -
3); Published by UH Bristol (University Hospitals, Bristol)}
Planning data collection
To make sure that the data collected are precise, and that only essential data are collected,
certain details about what is to be audited must be established from the outset. These are:
the user group to be included, with any exceptions noted

the healthcare professionals involved in the users' care
Agreement
the time period over which the criteria apply .
on these points helps to limit the amount of data that'have to be collected. It is

also important to decide on the type of data analysis thaf is to be used before data
collection starts .
Staff who are involved

reliability of information
in the care process that is to' beaLidited
that may be held about users on electronic
will know the range and
systems or in medical
records,
Consultation
and administrative staff may have access to further management
with these groups can help a team think through what data they really need to
collect to monitor those processes that directly affect care .
information .
Time periods are often specified

number of cases to monitor performance.
to enable data collectors to gather
The time period chosen depends on:
a representative

the numbers of cases that are treated on a daily basis
the number needed to make a confident judgement of care provided .
Sampling
Which
Once the
cases
group
should be audited?
or population of users has been precisely defined by specifying the
'inclusion criteria', it is time to decide on the records from which data will be collected.
may not always be practical or feasible to include each and every user, and in this case, a
representative sample is usually chosen from which inferences about the total population
It
can be made. When choosing a sample, two questions need to be answered .
1. How many of the users (study population) do I need to select?

2. How do I choose a representative sample?
Interactive workshop on c1inicaiaudit Page 57 of 117

The sample population will be dependent
treatment
upon the audit topic. Occasionally an aspect of
or care that cpplies to all patients is audited e.g. nutrition. However, the majority

of clinical audit tends to focus upon the care of a defined group of patients who share
certain characteristics. Typically the fact that they have the same medical condition,
received the same forTI of treatment or were seen within a certain time frame.
have
For

example, patients over SO years of age admitted to the CCU for a suspected MI.
In an ideal world the organization would audit the care received by all the audit population,

i.e. every patient
treatment
seen for a given condition over an extended period of time, every
received and every outcome achieved, in order to see whether their Care met the
agreed standards of best practice. However, if the number of patients in this population is

too large this becomes impractical and the audit team will need to look at a sample of the
overall population instead.
How many cases should be audited?
For research projects it is very important that a scientifically valid sample is selected. This is

because research is at its most powerful
population, nationally or even internationally.
method would not be adopted without
when its results are generalizable to a larger
For example, a previously unproven surgical
convincing evidence that it worked otherwise the

implications of a change in practice could be catastrophic. Clinical audit, however, simply
asks, 'what is happening here?' so the answer does not have to be as definitive as it would
need to be in research.

The sample selected for a process-based clinical audit project should be large enough so
that senior clinicians and administrators are willing to implement changes based on the

audit findings. It is important to be pragmatic, because research is not being done. In terms
of clinical audit projects a 'snapshot' sample is usually sufficient,
process-based audit. This will enable the audit team to measure whether
roughly 20-50 cases, for
processes are

being followed as per the standards set. Choosing a larger sample size than is necessary

takes up extra time and resources without adding value, and can mean that there is no time
and energy left within
bring about improvement.
the project team to address any issues of below-par practice and

It is <llso important
about improvement;
that the sample contains current or recent patients. Clinical audit is
the past C<lnnot be changed but the future can be changed. For

example if the audit project indicates that the patients seen in the previous month were not
given the right drug, changes can be implemented to ensure that future patients are. If,
however, the audit project indicates that patients seen three years ago were not given the

right drug, there is nothing that the organization
what constituted
can do about that now. It might be that
best practice three years ago was different. Rarely does the audit team
need to look at practice more than 12 months ago unless for a specific reason, usually


connected
procedure .
with outcomes rather than processes e.g. looking at outcomes of a rare
Whilst a 'snapshot' sample is usually sufficient for process-based audit, if the organization
requires greater assurance in the audit results, without looking at every patient in the
population, the audit team may need to calculate a sample size that is representative
whole population. This is likely to be the case when auditing outcomes, to be assured that
the results got are within the expected range .
of the

Sample size calculations depend on four variables:
Size of population .
Degree of accuracy required .

Degree of confidence required.
How often you expect your audit criteria to be met .
The following example shows how this works in practice:
An organization is planning an audit of the care of patients with hypertension. There are 300
patients (size of population) being treated for the disorder, but the clinical audit team do
not have time to review the records of them all. The audit criterion
receiving treatment
states that patients
should have had their blood pressure checked and the result below
150/90 on three occasions in the past 12 months. The target for meeting this standard is set
at 70%. However, the team are willing to accept 5% inaccuracy (degree of accuracy) due to
sampling. In other words, if the findings give a level of 70%, on 95% of occasions (degree of
confidence) the true value would lie between 65% and 75%. The public domain software
programme Epi Info (wlVw.cdc.gov/epiinfo) was used by the team to calculate the sample
I.
size using the above parameters, and the sample required is found to be 155 .
Strictly speaking, a sample size calculation should be carried out for each audit criteria that
is being addressed as part of the clinical audit project. The sample size chosen for the
project should be the largest figure that those calculations produce .
The table below appears in a number of guides to choosing audit sample sizes and assumes
an expected incidence of 50% i.e. that standards will be met 50% of the time. It gives the
sample size that will be needed in order to be 95% sure (degree of confidence) that the
results obtained from the sample will be within 5% (degree of accuracy) of the results
organization would have obtained for the whole population

of them. Put another
representative .
way, there
if they had collected data on all
is a 1 in 20 chance that the results will not be

Interactive workshop on c:inical audit Page59 of 117

50 44

100
150
79
108

200
500
132
217

1000
2000
278
322

5000 357
Table 2: Sample size

Using this table,
organization
if the audit showed that audit criteria X was met in 56% of cases, the
could be 95% sure that criteria X would have been met in somew here between
51-61% of cases had they looked at the whole population.

Remember, sample sizes can vary according to anyone of the following:
1. The expected incidence of the thing organization
2. The confidence
90%, 99% etc.
level required. The confidence
is auditing.
level does not have to be 95 %. It cou Id be

3. The level of accuracy the organization
be 5%,10%,1% etc.
is prepared to accept. The level of a ccuracy could

The table below illustrates how the sample size might vary for a population of 500:
dence level

Sampling methods
Table 3: Sample size

Once the audit team has decided to take a sample and have decided on th e size of that
sample, the next question is which cases are they going to include in the audit?


Sampling methods range from very simple to highly sophisticated.

audit projects use random sampling or convenien'ce sampling .
The majority of clinical
Simple Random Sampling
In a simple random sample every patient within the audit population has an equal chance of
selection.
random
This means that each case in the group
numbers table is used to identify
is allotted a number, and a published
the case numbers to include. Pocket calculators

and computers can also generate random numbers .
Simple random sampling is an example of a probability sampling method. It should result in

the sample being representative of the characteristics of the whole population, due to
random selection
selected group different
reducing the possibility of any systematic
in character from the overall population.
results this method should be used in conjunction
bias that would make the
To ensure representative
with a calculated sample size .
Consecutive Sampling
Consecutive sampling is often referred

next, or last however
to as convenience sampling. It involves choosing the
many cases, e.g. the next or the last 50 patients,
patients seen over the oourse of the previous or next month .
or alternatively, all
Consecutive
practical
sampling is an example of non-probability
way of selecting cases for a 'snapshot'
sampling
sample of the population.
and is often
However,
the most
it is

important to remember that the sample produced may differ in character from the overall
population and therefore the audit results may not be representative of the overall care
that is given .
Quasi random sampling is also referred to as systematic sampling. If the overall audit
population is 1000; the representative

1000 the organization
ensure that every patient in the audit population
sample would be 278. Since 4 x 278 is approximately
would select every fourth patient from the overall population.
has an equal chance of being selected, the
To
starting
between
point needs to be picked randomly. In this instance the starting
1 and 4. This means that you could be auditing
11, 16, etc. The start point must be random
patients
number must be
1, 5, 10, 15, etc., or 2, 6,
because if the audit team always started with
the first patient,

important
the second patient would never have a chance of being selected. This is an
consideration from a statistical point of view .
Stratified Sampling
Stratified sampling ensures that the proportion of different groupings present in the
population is reflected in the sample. For example if the patient population

75% men and 25% women, taking a simple or quasi random sample runs the risk of selecting
is made up of
Page 61 of 117

only men when it might be that there are particular aspects of care being audited which
relate specifically to women. So, if the overall population was 500 patients, this number

would need to be split in a ratio of 3:1 in favour of men, producing a ratio of 375 men: 125
women. This would result in the representative sample of 217 patients being split 163 men:
74 women. To select the random sample, separate men and women into two groups and

randomly select from both i.e. 74 women from a population of 125, and 163 men from a
population of 375.
Interval Sampling
Random sampling assumes that the sample can be drawn from a defined population of

users or cases. However, users do not form a static population, and the individuals making
up the user population (i.e. those attending clinics, practices or who are admitted to
hospital) will change during the audit. In these circumstances, the sample is often

determined by intervals of time; for example, people admitted to the coronary care unit
from January to March inclusive. This is a reasonable approach provided that admission
rates and the quality of care are not influenced by major seasonal factors.
Twa Stage Sampling
A small sample is selected first, and if unequivocal conclusions can be drawn, no more data

are collected. If the results are ambiguous a larger sample is selected.
Rapid Cycle Sampling

The traditional audit cycle often involves collecting relatively large amounts of data over a
long period, with a similar protracted data collection after changes are introduced. Although

this approach, if correctly applied, provides good information about performance, it can
make the process of change slow. An alternative involves the use of small samples, with
many repeated data collections to monitor serious fluctuations or changes in care. The cycle
is completed quickly, and reliability is improved by the repeated data collections.
Data Collection

Clinical audit is usually concerned with gathering quantitative data, sometimes referred to
as 'hard data'. Quantitative data is numerical data that is used to measure variables e.g.
counting the number of times certain things are done, how often the" are done and to what
end. In clinical audit this data is linked to standards of best practice which define what
should be done, how oft2n it should be done and what outcome is expected.


Depending on the audit to pic, the organization
data. This is typically
might also want to collect some qualitative
achie ved by using a survey to capture patients' or staff experiences or
opinions. Qualitative
What data needs to be collected?

data is subjective and is sometimes referred to as 'soft' data.
Data should only be collec ted that is necessary to satisfy the audit aim. The data items that
are collected should ena ble the organization to measure practice against the audit
standards.
considered
It is best to a void collecting superfluous data items, just
to be interestin g or useful, as this will result in more time spent on the project,
but with little or no additio nal benefit .
because they are
A useful technique to ensu re that the data items are necessary is to use a data matrix. The
data items are listed in th e first column and a tick is placed in the box (es) according to
which standard(s)
are that the organization
it relate s to. If there is no tick against a particular
s hould not be collecting
that the data item is impor tant, the organization
it. Alternatively,
data item, the chances
if the organization
might want to consider whether
feels
there is a
need to rewrite an existing standard or include an additional standard .
Data item
Sex
Standa rd 1
v
Standard 2 .
v
Standard 3 Standard 4
Age
Table : Data matrix table
Where will the data be found?
If data is routinely recorde d, either in the patient's notes or electronically, it is possible to
carry out a retrospective a udit, I.e. assessing past episodes of patient care. If the data is not

routinely recorded pro>pe ctive data collection is required i.e. assessing patient care at the
time it is given .
Both methods have their advantages and disadvantages. Retrospective audit can be quick
but can turn into an audit of 'how well is care documented',

g iven'.
otherwise
Pros p ective audit can be used to gather
have access to but it may have problems
data that
rather than 'how well care is
the organization
ensuring that the audit team capture
would not
every
problem
case particularly
can be mitigated
verbal information
if other
through
people
on the implementation
are completing
the provision of training
of the data collection
the data collection
accompanied by written
forms.
process prior to the start of

This
and/or
the audit. Additionally
would assume that compliance

prospective
they are being audited they might alter their
data collection
with the standards

might be affected by bias, if people know
behaviour
audited
accordingly, i.e. the organization
would be higher compared to
the results obtained if people were unaware that they were being audited .

Prospective

Definition Data collected by looking back Data collected from this point
onwards, or over the practice
starting at a future date
When to use When looking at what has been

happening in a chosen topic area
When data are currently unavailable
(i.e. has not been recorded as part

of daily practice)
When data are of a very poor

quality/incomplete and it is
therefore not possible to audit
Advantages Can be faster Avoids using poor-quality,

incomplete data
Allows design of a clear and concise

Disadvantages
Provides a baseline
Past service users do not benefit

data collection sheet
Provides no baseline for the audit

Data may be difficult to collect (e.g. Can be time-consuming
poor-quality, information missing) number of individuals
since
must be
a

relied upon to collect the data
Table : The differences between retrospective and prospective data collection

Adopted from: ep by
Royal College of Psychiatrists

Who will collect the data?
It is usually the responsibility of the audit team involved in the project to undertake the data

collection ..
However, staff not directly involved in designing the project are often asked to collect data

on behalf of the audit team. For example, nursing staff might be asked to complete data
collection forms for the patients in their ward. If other staff members are going to be
responsible for the data collection process they will need to receive training beforehand.


Designing a Data Collection Form
The audit team will now need to design a data collection form to obtain the audit data. In
clinical audit, data collection forms are also referred to as 'audit forms' or 'audit proformas' .
The word 'questionnaire' is usually reserved for surveys, where patients or staff are being
asked questions. Qualitative methods of data collection such as focus groups and in-depth
interviews sometimes generate

employed as part of a clinical audit itself .
ideas for clinical audit projects, but are not usually

The term 'audit tool' is a generic term covering any form or system used to facilitate the
clinical audit process .

The design of the data collection
following:
form is very important. It is important to include the
This ensures that the form is easy to identify

if it is lost or to retrieve once it has been archived at the end of the project.

and contactable if there are any questions
or where to return it. Again this information
This ensures that the lead is easily identifiable
or queries about how to complete
also helps to link the form to a particular
the form

audit project.
If the form is to be completed by someone

other than the audit team, clear instructions should be provided for completing the
form, such as 'tick the box' or 'circle the appropriate answer'. If clear instructions are not
provided people will not necessarily record the information that is required or they
might record information that in ambiguous.

team might not be able to analyse the data easily or exactly .
This might affect the results as the audit
If someone else is responsible for the data

collection clear instructions
not applicable,
for returning the form are vital. The need for instructions
however, if the audit team is doing the data collection
AII date items must be included. It is important to remember
itself .
to include the
is
data items
information
relating to the identified exceptions.
detailing why certain standards were not met.
Exceptions will, in part, provide
Other issues to be aware of when designing your data collection for are that:
./
./
Questions should be clear and unambiguous .
The format that the organization
Time of admissiQn : (24 hour clock)
wants the data recordedn is clear. For example,
./ Questions shou:d be well spaced out. Importantly whilst it is important

the form, be careful not to use a font size that is too small to be legible .
not to clutter

./
./
If the audit team wants opinions or views, allow space for these to be recorded.
Keep the data collection form as succinct as possible.

confidentiality issues

Once the audit team has collected the data, they might want to be able to identify which
data collection form corresponds to which patient. However, in order to ensure patient

anonymity, personal details, including name, address, data of birth etc., should not be

recorded on the data collection form. Therefore the best approach is to number each of the
forms using a unique identifier. A separate piece of paper, or 'code sheet' should then be
kept, which links each unique identifier to the patient's hospital number. Without this list,

data collection forms cannot be linked to specific patients.

Occasionally it might be necessary to record patient identifiable
collection
collection
form.
form
For example, during
might have to follow
a prospective
a patient through
information
data collection
their
on the data
exercise the data
pathway of care. In this

instance the only patient identifiable
their hospitallD
information that should be recorded on the form is
number. Name, date of birth, etc. should never be recorded.
Other data protection
./
issues to bear in mind are:
Data collected must be adequate, relevant and not excessive .

./
./
Collected data should be kept securely, so that members of the public cannot access
it. This relates to both electronic and hard copies of data.
Data should not be kept for any longer than necessary. Completed data collection

forms should be destroyed once the identified retention period is completed.
Pilot the data collection form

Before collecting the data for the entire sample it is important to pilot the data collection
form. The purpose of the pilot is to tryout the data collection form on a small sample to
make sure that it works, especially if someone else is going to be collecting the data for the

audit team. The pilot may reveal that some of the instructions on how to complete the form
or the questions asked are ambiguous, that the form is difficult to complete, or that the
team is simply not getting the information they wanted. It is essential that the team analyse
pilot data against the standards to ensure that they can measure current practice against
them. This is the time to put right any problems so that when the audit is done properly, the
team will end up with the right information, first time.

The pilot test might also indicate where open questions can be reconstructed
options. It is suggested that where possible avoid using free text on data collection forms.
as tick-box

This kind of data can be complex to analyse and is more suited to the kind of qualitative

work, e.g. focus groups and in-depth interviews, which may precede an audit.

Data Analysis
Objectives
When analysing the data the team will generally want to try to reach conclusions about:

the general pattern of actual practice
the degree to which actual practice (results of audit) is meeting the standards set
those cases for which it is clinically acceptable for the standards not to be met.
Methods of data analysis

Analysing audit data does not usually require complex statistical tests, although these may
be necessary in certain situations. Clearly the type 6f data collected will determine the type
of analysis employed. The following approaches may be used in analysing the data .
(A) escriptive statistics
This is where the data are described numerically. The teammay wish to calculate:

the frequency of certain events/values occurring (i.e. rates and percentages)
the mean, and/or the median - the most 'typical value' for the data
the range and/or

results .
standard deviation - to show the variability of the individual
It is often useful to present descriptive

charts .
(B) tatistical tests

statistics graphically using, for example, bar or pie

These may be used:
when 'conducting an outcome audit, for example comparing 'before' and 'after'

results on questionnaires to find out whether there has been a statistically significant
improvement in the patient symptom scores; or
when wanting to show whether the results obtained can be attributed to chance
(C
variation .
ualitative analvsis
Where
qualitative
open-ended questions
data will be obtained.
have been asked as part of the clinical
There are a number of ways of analysing qualitative
audit project,
data .
It may be possible, for example, to conduct a content analysis of the major recurring themes
and a frequency count may then be performed .

Comparing with standards set

Where
percentage
standards have been set, the final part of the analysis will entail
of cases meeting and not meeting each standard.
calculating the

At the standard-setting
possible to identify
certain
situations
standard.
and design stages of the clinical audit cycle it may not have been
circumstances
Discussions with
in which it would be acceptable
colleagues
in which it is considered clinically acceptable
about specific
for cases not to meet a
cases may highlight
for standards not to be met. In these
some

situations, the results may prove most meaningful if you calculate the following
percentages:
percentage of cases meeting each standard (calculated from whole sample including

'.
non-applicable cases)
percentage of cases not meeting each standard (again including non-applicable
cases)
percentage of cases considered non-applicable (not meeting standards for clinically
acceptable reasons)

percentage of applicable cases meeting each standard
percentage of applicable cases not meeting each standard.
Where there is only a small difference between the target set and the percentage of cases
meeting
whether
the standards
due to chance. Confidence
there
in the clinical audit, it may be difficult
intervals
is a statistically
can be calculated
significant difference
to obtain
between
to know whether
a more accurate
your results
this is just
idea of
and the set

standards.

Just as the analysis should be as simple as possible, the findings should be presented
and clearly.
simply
Generally when displaying continuous
you to layout this numerical
data, it would be appropriate to use charts that allow
scale and plot data against it e.g. scatter graphs, box and

whisker plots, etc.
The usual type of data resulting from a clinical audit is categorical data and would most

commonly be represented as a bar chart or a pie chart. Bar charts have become the most

common format, but the numbers should be available in separate tables rather than
presenting the charts alone. Good charts should focus on getting the message across, rather
than creating fancy, and distracting, images. Clutter should be avoided and the charts clearly

labelled.


Summary
Sample
o Identify the population characteristics.
o Find out how many patients make up the population .

Decide whether to look at every case or choose a sample .
o Process based clinical audit projects usually involve a 'snapshot' sample, of roughly 20-
50 cases .
o
If the audit team has calculated
randomly.
a statistically representative sample size, select cases
Talk to colleagues who will have the final say about any changes in practice about how
many cases they would

resources and requirements
like to see included
for statistical validity .
in the audit and weigh this against time,
o
A small, convenience
Beware of potential
sample of current cases may be all thafis
bias in results .
needed .
o
Do not collect excess data or patient or staff identifiable
Always pilot data collection forms .
information.
o Type of analysis to be used should be identified at an early stage, as it influences both
o
the type and amount of data collected .
Analysis
sophisticated
can range from a simple
statistical techniques.
calculation of percentages, through to relatively
o On most occasions, however, simple methods

are to stimulate
are preferable,
change, the analysis must be simple enough for everyone
process to understand .
and indeed, if the results
in the care
Acknowledgements
1.
2.
Principles for Best Pr3ctice in Clinical Audit: NICE guidelines published in 2002
Undertaking
Psychiatrists
a clinical audit project: a step-by-step guide; Published by Royal College of

Where will the data~befound?

Ideally all the necessary data items would have
been collected routinely and be readily
accessible.

Reality:

- data are often held in several databases, on
paper or electronically,
- in different departments, even different

organizations, or may not be collected at all.

Where will the data be found?

Check if existing record and information
systems may already be adequate for
clinical audit purposes
Design of specific electronic encounter sheet

or data sheet may be required

Worksheet preparation and
methodology of administration

Simplest for the purpose

Only essential data is collected
Probability of bias is to be considered
- Non response to a survey

- Unavailability of certai.1 type of case note
- Selective referral of certain types of patients
- Failure of patient to tu-n up for follow up


Audit form or proforma
Audit project title and date of the audit

Contact details of the project lead
Instructions for
- completing the form
- returning the form .

Data items .

2.
opic chosen by you in exercise

' .


-..


NOTES
, .

'
.

NOTES

Stage Four:
{Adapted from - How To:

M~\::"\l'mprovements
- Clinical Audit Report &
Presentation (Version 3)
by UH Bristol (University Hospitals, Bristol)}
; Published

"The most important part of the audit cycle is making change"

Baker et al (1999)
The aim of this section is to provide advice on how to implement change successfully. If an

audit shows that current practice needs to be improved, making changes is important. The
public has the right to expect that practitioners will provide care that is consistent with
recognised good practice. However, it is important to bear in mind that not all changes are

necessarily improvements. Do not make changes for change's sake. At an appropriate time,
repeat the audit (re-audit) to ensure that changes have been implemented and that practice

has improved.
Communicating Findings
The first step in making improvements is communicating the findings to the relevant
'.'.
e,
stakeholders. This is an important part of the clinical audit process if it is to have any impact
on the quality of the service that the organization is providing.

Who should know about the findings?
It is important that all of the key stakeholders are made aware of the findings of the project

and are provided with an opportunity to comment on them. This will include those
individuals:
.-
whose practice was examined
who are on the clinical audit project team
who would be involved in making changes to improve the particular aspect of care in
question.
Different people may have access to different levels of information. For example clinicians
may know the 'scores' of all of the other clinicians if previously agreed, but it may not be
appropriate for outsiders to have accessto this level of detail.
'.

A combination of passive feedback (written information) and active feedback (discussion of
findings) is preferable when communicating the findings of your project.

Interactive workshop on clinicalaudit Page 76 of 117

.
.
'
":.-
Audit reports (passive feedback)

It is important to produce a written record of the clinical audit project, which clearly
outlines how the audit team approached each stage in the clinical audit cycle and the results
obtained. This can then be disseminated to the relevant people as a way of feeding-back
findings. This also ensures that a record of the study is kept for future external and internal
use, for example by individuals wishing to conduct a similar clinical audit project.

Discussion of results (octive feedback)
Discussing the results of the clinical

exercise through
audit project with key stakeholders
which areas of practice which needs to be changed can be identified
is an essential
and
agreed .
'.
.
How to Write the Clinical Audit Report?

Name of the organization
Project title
and name of division/specialty

Project lead/s (and name of the person who wrote the report, if different)
Date of report
This section explains the rationale for doing the audit, i.e. why it is a priority for quality

improvement. The evidence base for the audit topic should be summarised, with full
references provided at the end of the report. This is a good point to explain how the team
was organised and who was involved .

.- This section sets out the aim, objectives and standards of the clinical audit project.
'.

Aim - Defines what the organization
project.
Objectives
the aim .
- Defines the individual
hoped to achieve i.e. the overall purpose of the
steps that need to be taken in order to achieve

Standards - The quantifiable statements detailing the specific aspects of patient care
and/or management that the team measured current practice against. The audit
criteria, target, exception(s) and source(s) of evidence should be specified .

Page 77 of 117

.
,

This section should outline:
The population fDr the audit project. For example:

o "Patients aged over 50 years Df age admitted tD the CCU for a suspected MI".
Whether it is a retrospective or prospective audit. For example:
o "A prospective audit assessing the first 30 patients aged Dver 50 whD were

admitted tD the CCU fDr a suspected MI from 01/03/2011".
o "A retrDspective audit 10Dking at all patients aged over 50 whD were
How
admitted
these
tD the CCU fDr a suspected MI during February 2010".
patients were identified, e.g. from the electrDnicrecord system,

labDratory systems, radiology database etc.
Sample size.

Time periDd audited.
The data cDllectiDn method. FDr example:
o "Data was collected from patients' case nDtesusing a data collection form
.-
o
(see Appendix A)".
"Patients
consultation
were asked
(see Appendix A)".
to complete a patient survey fDllowing their

Who was responsible fDr data collection?
The method of data input (if appropriate) and analysis e.g. data was input into and

analysed using Microsoft Excel.
The results for each standard should be presented in this sectiDn to establish which
standards
standard
are being met, and which are not. If the audit team finds a standard is not being
met they need tD identify
is met in the future.
why and how practice
The team may also consider
can be improved to ensure that
if there were other, acceptable
the

reasons for the standard not being met, i.e. an exception
stage.
not considered during the planning

The results to each stcndard
appropriate.
should
In the clinical audit report and presentatiDn

be presented using graphs to further
the audit team should state how many patients

illustrate, if

were included in the audit; this is the initial 'n' number. If the data is incomplete the reason
(s) for the same need to be explained,
notes.
e.g. the team was unable to find every set of patient


i

.- Include both the number and percentage of cases meeting each standard. Make it clear
what number the team is taking a percentage of as the 'n' number may change for each
standard, e.g. 45/50 (90%) .
Data may be presented as pie charts or bar graphs. Be selective

illustrate
appropriate
the key findings in this way so as not to overburden
chart for each piece of data, e.g. pie charts to show proportions,
in the use of charts; only
readers. Use the most
or bar charts
for easy comparisons. Quote both raw figures and percentages in the chart where possible.
Where the team can only have one or the other, pie charts should have the raw figures as a

label by each segment, rather than percentages, as this chart is designed to visually show
proportions (percentages). Charts showing only percentages should be accompanied by a
table showing the raw figures; these will be needed when it comes to "Ore-auditing and

comparing results .
Individual health care staff should not be identifiable ih the report. Clinical
-. audit should not be used as a 'witch hunt'. If, for example, the audit iscomparing
of three consultants, th2y could be called as A, Band C. For confidentiality
the results
reasons it is also
important that patients are not identified .
-.- This section should list the key points that flow from the results. The team should use bullet
points and avoid using long paragraphs.

data, or if the data points to no firm conclusions,
Ensure that the conclusions are supported
say so. Never make claims that are not
by the

supported by the evidence. Make objective, factual statements, not subjective ones, i.e. do
not say 'it is obvious that...' or 'clearly, what is happening is..: .
Where appropriate, recommendations for change should be made. Make sure these are
. -
realistic and achievable.
should have already investigated
If money is needed to implement
whether
the recommendations,
there are suitable funds available .
the team
The action plan should be finalised after the team has presented the project e.g. at a clinical
audit presentation meeting. It is as a result of this presentation that the recommendations
will either
preferably
responsible
be accepted
at that
or revised.
meeting, to confirm
and when they will be implemented
Following
what
this, an action plan should
changes will be implemented,
by. The action plan should be included
be agreed,
who will be
in
either the body of the report, or if the report has already been completed,
to the full
complete
report. If appropriate
the audit cycle .
a date for a re-audit should
as an addendum
be included in order to

,.

Full references for any literature discussed in the report's background section as well as for
the source(s) of evidence that the standards are taken from must be provided. It is
important
journals,
to be consistent
numbers references
in the referencing. The Vancouver Sl'(le, favoured
in the order in which they appear in the text. For each
reference provide the names and initials of all authors followed
by many
by the title of the article,

the title of the journal, the year of publication,
page numbers.
the volume number and the first and last
References to books should give the names of any editors, place of

publication, publisher and year.
A copy of the data collection form should be included in the appendi~.

Clinical audit reports should be written up in sufficient detail for a reader, who did not

conduct the audit, to be able to ascertain how the audit was conducted.
./ Think about who is going to read the report and gear the content and style accordingly.

With this, also think about what the organization
needs to persuade people
persuasively and puts a good case .
of the need for
is trying to a:hieve e.g. if the team
action, make ,ure the report reads
./ Make it look professional and interesting. Use colours and fonts sensibly, be consistent.

./ The report should be written in plain English and have a logical flow to it. Make the
structure explicit with section headings and paragraphs. Use page numbers.
./ Every word in the report should count for something. Do not embellish your report
unnecessarily e.g. 'With this in mind ...' or 'All things considered ....' An audit report
should be largely descriptive,
responses to open questions
i.e. a statement
it might
of fact. However, when analysing the
be appropriate to include the opinions and

./
anecdotes of the respondents.
When using abbreviations and acronyms it is good practice to write these out in full in

'.
the first instance .
./ It is good practice to ask someone else to proof read the report before distributing it.
They can check for errors, ensure that the report is easy to understand and that it flows
well.


Changing Behaviours
Change can occur at organizational,
a
group, or individual levels .
Organizations
forces are
can be thought
opposed
of as existing in a state of quasi-equilibrium,
by restraining forces, with the net effect that
in which driving
changes in the

organization are minimised. The status quo must be 'unfrozen' to allow change to occur,
followed by 'refreezing' to consolidate the new equilibrium. In order to create the
imbalance
restraining
between the driving
forces should be selectively
and restraining forces that is needed for unfreezing,
removed or reduced. Merely increasing the driving
the

forces will stimulate an increase in the number or strength of the restraining forces .
Driving forces

Restraining forces

Theories of social influence and conformity can be used to explain change within groups
that are smaller than whole organiZations. A group exerts pressure on its members to
conform,
1999). However,
and this press~re can be so powerful
believe that the group view is valid and adopt
a minority can influence
that individuals
it as their
the group if it has a powerful
not only comply, but come to
own (Hayes, 1994; Robertson,
role and makes its
case consistently. The minority may increase its power through seeking allies from inside or
outside the group .
The trans-theoretical
addictive behaviours,
change as a transition
model, which
such as smoking
was developed
(Robertson,
through a series of five stages:
for management
19991, explains
of
individual
people with
behaviour
1. pre-contemplation - the individual has no intention of changing
2. contemplation - change is regarded as a possibility in the near future
3.
4.
5.
preparation - explicit plans are made
action - the change occurs
maintenance - the changed behaviour is consolidated .

Progression through
encourage someone
each stage is necessary if a change is to occur. No single strategy
to progress from pre-contemplation to maintenance, and different
can

strategies are required at each stage to help a person move on to the next.
change

Theories of behaviour
preparing plans to improve
change are helpful, but they must be set in a practical framework
care. The concept of barriers to change is a feature of many
of

models that draw on different theories of behaviour change. If the barriers to change are
identified beforehand, implementation methods can be tailored to overcome them
(Robertson
One relatively
et aI., 1996).
practical framework that incorporates the concept of barriers to cha nge has
five principal steps (Grol, 1997).
1. The required change is clearly defined, based on evidence, and is presented in a way

that staff can easily understand.
2. The barriers to change are identified using some of the methods mentioned below,
including those relating to professionals
a. Interviews of key staff and/or users
and to the healthcare organization.

b. Discussion at a team meeting
c. Observation of patterns of work
d. Identification of the care pathway
e. Facilitated team meetings with the use of brainstorming or fishbone diagrams
3. Implementation methods are chosen that are appropriate to the particular

4.
circumstances, the change itself, and the obstacles to be overcome. An understanding
selected theories of behaviour change may be used to inform the choice of methods.
An integrated plan is developed for coordinated delivery and monitoring
of
of the

interventions. The r:lan should describe the sequence in which interventions will be

5.
made, the staff and resources required to make them, and the target groups.
The plan is carried
additional interventions
out, and progress is evaluated,
being used as required.
with modifications to the plan or

This model and others
must be carefully planned
like it make clear that implementation
and systematically managed.
of change is a process that
The particular interventions or

implementation methods used form only one aspect of the process of improving care.
Implementing Change

Clinical audit is recognised as an effective
improvements in patient care, management
is easy. It is the most difficult
means of changing clinical practice to bring about
and outcomes; this does not mean that change
part of the clinical audit cycle, and the point at which projects

are most likely to lose momentum.


To maximise the likelihood
outset, to:
of change the clinical audit project should be designed, from the
Ensure staff are motivated to improve

else, or if the audit is being done simply
practice. If the audit does not interest
because it has to be done, the chances of
anyone

bringing about change are less likely .
Involve all the key players. If all of the people who will have the final say about changes
in practice are involved with the project from the very beginning the likelihood that the

proposed changes will be agreed and implemented
If there is additional costs associated with the
will be increased .
proposed change, ensure that

management understands and support the proposal. If the team does not get this
agreement before starting the project, it is less likely that they will be able to get the
funds required to make the change .
Use robust methodology in the project. If people are confident in the validity and
reliability
results .
of the results they will be more likely to make the changes indicated by the
Be aware that change may be perceived positively
The change process

or negatively .
There are three main stages to the change process. These are summarised below:
1. Initiation - The process leading up to the change .

2. Implementation - The first experiences of change .
3. Continuation - The changes become embedded .
In the sections below we will understand the key elements of initiating change and
implementation. In the next chapter we will understand continuation .
The situation will have to be analysed before suggesting changes .
Do people recognise the need for change? The presentation

should be used to notify
recommendations
of the audit
people of the need for change and to 'sell' to them
for change .
results
the
Sometimes people will readily recognise the need for change, perhaps there
been a series of critical incidents in a particular
there may be a need to highlight the importance
area, whereas on other occasions
of change.
have
Willingness
been working
to change varies from person to person. For example, someone who has
in a particular clinical area for a short period of time might be more

open to the idea of change than someone who has been working there for a longer
period.

The proposal may have to be sold. An important
clinicians will be interested
patients, they will probably
factor to bear in mind is that, whilst
in what the proposed changes might mean for their
be most concerned about the implications for them

personally.
People respond to different stimuli when it comes to thinking about change. For
some, a shared vision of the future will suffice. Others will want to be persuaded by

facts and figures.
There may be individuals who will only change practice if a reward or penalty is at
stake. Use power or influence where it is possible.

The majority of any group will accept changes in response to the action of opinion
leaders i.e. people who are well respected. It is therefore
leaders on the side of the audit team; this is particularly
important to have opinion
true if potential
the proposed change are cultural ones relating to existing routines or practices.
barriers to

The implementation phase will have to be planned,
purchasing a piece of equipment.
even if it is simply a question of

The change may have to be broken down into manageable tasks and achievable targets. j
Crucially, it also means communicating e.g. informing staff about what is going on and
consulting them for their own ideas. Different objectives require different methods of

communication. Sending out a newsletter about a new clinical guideline is, on its own,
unlikely to change clinical practice. It is therefore important to consider whether or not
there is a need for training and development, e.g. organising briefings/ workshops.
The organization could decide to pilot the change, e.g. implem'ent it for a fixed period of

time before
demonstrate
reviewing the situation; this is particularly
the benefits to previously unconvinced staff.
important if you need to

Reasons why change sometimes
Change can fail at anyone

fails
of the stages outlined above. The main reasons being a lack of

resources, a lack of motivation, inadequate management of the process, or poor

communication.
Usually factors that hinder change can be addressed by planning the clinical audit project
properly:

.'

Create a multi-professionaljmulti-disciplinary
each staff group involved in the care being audited;
audit team with a representative
this will increase ownership
from
of the

problems and improve motivation
Ensure you have involved people with authority
for change .
If it is likely that resources are needed to implement

to agree changes .
changes, ensure that management
is in agreement with the aims of the project and will provide funds if necessary .
Promoting Successful Audit
Factors that promote

sound leadership
the success of clinical audit include:
a conducive/supportive organizational environment

structures and systems to support audit, including mechanisms to make data

collection easier
a well-managed audit programme
addressing a range of issues important to the organization and individual clinicians
giving adequate attention to all stages of audit (Buttery et aI., 1995b; Rumsey et aI.,
1996a; Johnston et aI., 2000; Rumsey et aI., 1996b) .

The environment in which audit is performed
a culture that supports them. Indeed, as change is so dependent
needs the appropriate structures
on behavioural
in place and
factors, the
nature of the environment

aspects of audit.
Individual environments
is even more important for implementation than for other
Individuals
role in
need time to devote to implementing
planning the changes. Giving people
improvements,
an opportunity
even if they do not have a
to think through the
implications,
challenge.
and to discuss practical
Time alone is not sufficient,
available to help individuals
issues with
however,
others, can make change
and systems and support
improve their existing skills or develop new ones .
less of a
must also be
The individual
quality
healthcare professional should also be able to report
of care, and be able to suggest new ways of working
concerns
that lead to improvements.
about the
Individuals
1999). An individual
are most likely to be committed
mistakes or poor performance

who is fearful of review
will be reluctant
and involved
and the consequences
to participate
when they feel safe (McCrea,
that might
in audit, and there is likely to
follow
be covert resistance to change. A culture that stimulates

feeling of safety created by actions that generate trust. Even data collections
below-average or poor performance can provide opportunities
commitment to audit and the
for building trust .

that highlight
.'

Team environments

At a very basic level, te3ms must be able to meet together,
share ideas about making changes in their work. Once meetings are established,
to discuss their objectives and
leadership

skills are needed to establish effective
contribute (McCrea, 1999).
communication and ensure that everyone is able to

Structure
Teams

,/ Time
,/ Personal development
,/ Leadership
,/ Clear and shared
,/ Explicit
clinical
commitment
audit within
to
the

plans objectives organization
,/ Access to advice about ,/ Effective ,/ Staff with responsibility for
change management communication audit are fully trained and
,/ Access to a system for ,/ Training in encouraged to develop new
reporting
,/ Occupational
concerns
health
improvement
,/ Opportunities
methods
for the
solutions to old problems
,/ Good systems for

service available team to meet to share understanding the views of
ideas and develop plans users
,/ Good communication with
other
agencies
health and social care

Culture
,/ Positive attitude to ,/ Open to new ideas ,/ Users' perspectives

audit and ,/ Focus on the user's genuinely regarded as the
improvement experience focus of quality improvement

,/ Lack
change
of
confronting
desired
fear
and
-
less than
or even poor
of
of
,/ Inter -professiona I
respect and cooperation
,/ Open
of
to interest
extern31 agencies
performance,
afraid of inspection
from
in quality
and not

performance ,/ 'No blame' approach to
errors
,/ Audit given a high priority
promote clinical audit

.
'
Successful
around
quality improvement requires
key aspects of care. Teams tend to focus on their
teams to build commitment
own internal
and motivation
concerns, and

systems are needed to enable them to understand

the user's experience, which is of central
Page 86 of 117

f- .

'
importance.
complete
Information from a user surveyor
view of its role and duties, and with effective
forum can help a team to develop a more
leadership, generate the motivation
for change .
Teams require effective leadership,

a team (Brearley, 2000). Leadership
but people do not automatically
skills can be developed,
have the ability to lead
however, and access to
leadership programmes should be available. Effective leadership establishes respect and
understanding
performance
between different
to replace defensiveness.
professional
Organizations
groups, and can help
also have an important
openness about
role to play in

this process. Managers also need to be aware that teams can become dysfunctional through
poor leadership .
Group and individual

implementation
factors
of change will be .
and leadership interact to determine how successful the

Organizatianal environments
Experience of audit in past years has shown that explicit objectives,

managing audit, and adequate support in terms of staff and facilities are needed (Buttery et
clear systems for
aI., 1995a). In addition, effective quality improvement depends on the involvement of users
in identifying
If an organization
issues for audit.
seeks to improve the quality of care through audit, it should regard the
experience
external
of service users as the starting point for change. The organization
review by the health service or the public, and must be able to show by its actions
must not fear

that its statements about openness and freedom from blame are genuine .
Summary
All clinical audit projects should be both written up as a report and presented verbally .

The finalised report acts as the official record of the project.
The report should include all the information
Both the report and presentation
needed to plan are-audit .
should include:
1.
2.
3.
Title Pagel Slide
Background
Aim, Objectives and Standards
4.
5.
6.
Methodology
Results
Conclusions
The report should also contain:
1. Action Plan
2.
3.
References
Appendices

o The presentation should be used to get the message(s) of the clinical audit project
across to key staff and should generate discussion and agreement about changes to

o
o
practice in light of the audit results.
Get people on boarc with the proposal.
Write and implement an action plan.

o Consider piloting change first and review.

oRe-audit to confirm improvement.
Acknowledgements
1. Principles for Best Practice in Clinical Audit: NICEguidelines published in 2002

2. Undertaking a clinical audit project: a step-by-step guide; Published by Royal College of
Psychiatrists


Action plan components
what needs to change

how change could be achieved - what actions

need to take place
who needs to take these" actions
when the proposed actions will begin
how these actions will be monitored and by
whom
how and when to assess whether the actions
taken have achieved the desired outcome
Organizational change
Restraining forces
Driving forces
./ Patient
pressure
./ National policy
I~
I<!--
Individuals
./ Fear of increased
workload
,/ Concern about
staffing and milo:

./ Demands from
../ loss of control
referring GPs
over work
for improved
1<!--

patterns
access
Organizations
..r Resistant culture
../ Lack of resources
../ Rigid structure

Group change
Theories of social influence and conformity
Pressure of group conformance
Engagement
Owning the process

Accountability with the outcomes
Haye.5. 1994; Robert'Of), 1999


Page 89 of 117

Individual change

Pre-contemplation - the individual
intention of changing
Contemplation - change is regarded as a
has no

possibility in the near future
Preparation - explicit plans are made
Action - the change occurs

Maintenance
consolidated
- the changed behaviour is

Implementing change
Change or else change will change you
No one shoe fits all

Diagnostic analysis to be undertaken before
selecting the most appropriate strategies for
implementing change.
,,..,, , SulMtlcllld, ZOOJ

Implementing change

~.'~ "..m;L"'''''''~~1l''~
-, ',". ' . 'f!l.Ii'J _.' -' --
..

_,.,-..,...".n.a.~~~.~~ho""tfJi"""'--t,..,....-not>

A~._I~
N_ee-.. _._


------------------------------,
--- ------_ .. -

Exercise
Kindly identify 5 barriers to successful audit .

NOTES


i
'.
. ~--

NOTES

".,.. .
.
~
' ...-

-~
x;
..
~
'
/

~.
'..
.

"

.
'

,f
'.' Interactive workshop on clinical audit ,. Page 93 of 117

Stage Five: Sustaining Improvement
{Adapted from - Principles for Best Practice in Clinical Audit published by

Although improving performance
is also essential. Indeed, any systematic
include plans to:
is the primary goal of audit, sustaining that improvement
approach to changing professional practice should

monitor and evaluate the change
maintain and reinforce the change

Monitoring and Evaluating Changes
Even if the organization manages to get changes implemented, it does not mean they will
stay implemented. People sometimes slip back into the old ways of working. Collecting data
for a second time, after changes have been introduced,
maintaining
selection,
the improvements
information collection, and analysis
is central to both assessing and
made during clinical audit. The same procedures
(see Stage Three: measuring
of sample
level of

performance)
comparable
only absolutely
should be used throughout the process, to ensure that the data are valid and
with each other. Rapid-cycle data collection may also be appropriate,
essential data are collected from small samples (again, see Stage Three).
in which

If performance
plan or additional
targets have not been reached during implementation,
interventions will be needed.
modifications to the

The clinical audit cycle may need to be repeated
confident that improvenents
several times before the organization
have been made to the quality of the service, and that these
is

improvements are being maintained.
The organization may dEcide that it would be more appropriate to conduct more specialised
clinical audits as a result of the first clinical audit project, rather than attempting to re-audit
the whole topic area at one time. The way in which you decide to approach the re-audit will
depend on the findings of your first clinical audit, as shown in Figure 5.
Once again, communication
a positive impact, through
changes, is management
is crucial. Provide staff with evidence that the changes have had
a re-audit.
encouraging
If other staff are slow to come on board with the
them to move their position? After all it is not only

important
side.
to keep people informed, the audit team also needs to keep management on

Other tools for sustaining improvement
1. Clinical performance indicators

are:


2.
3.
Monitoring
Monitoring
errors
adverse incidents
4. Significant event audit
All standards were

YES Repeat clinical audit process at a later
met date to ensure this is maintained
NO
Some standards After action has been implemented,
were
not met
YES
repeat data collection only for those
standards not met (in the short term)
NO
All/most After action has been implemented,

standards YES
repeat entire clinical audit process
were not met

NO
Conclusions could
not be reached - YES Review and modify standards - repeat
standards unclear entire clinical audit
auditing
Adapted from" by step guide"

Royal College of Psychiatrists
Maintaining and Reinforcing Improvement

Maintaining and reinforcing improvement over time is a complex process. In UK projects in
which improvements have been sustained, some common factors have been identified
(Dunning et aI., 1999), including:

reinforcing
continual
or motivating factors
cycle of quality improvement
built in by the management to support the

integration of audit into the organization's
strong leadership.
wider quality improvement systems

In quality improvement initiatives in US hospitals, four interdependent
found to support the lasting impact of clinical audit:
processes have been
a strategy
improvement
that recognises audit activity, combined with an achievable
a culture that supports the concept of planned audit activity, leading to improvements
quality of which everyone in the organization is aware and supportive
plan of quality
in

-.
IT processes that can provide accurate information about the organization, allowing
sensible decisions to be made about where audit is needed and whether changes have
had the desired effects

appropriate structures to support and implement the changes that are suggested
(Shortell, 1998).

: Developing and encouraging sustained quality
performance by healthcare organizations involves attention to three specific areas of
organizational development (Davies and Nutley, 2000; Greenhalgh, 2000; Huntington et aI.,
2000):
1. cultural change, ensuring that the shared values and beliefs of the organization

support the ideas of quality improvement
2. adequate training, so that staff can gather and analyse data accurately
3. an organizational structure that coordinates and monitors quality improvement work
reporting
quickly and effectively.
of audit into
An organization
an established framework
may benefit
that
from
already
fitting the planning
operates at strategic,
and

organizational, and clinical levels. The responsibility for providing education, training, and
support for audit teams lies at the organizational level.

Leadership plays a vital role in any process of quality improvement.
is not just telling people what to do, but also ensuring that the right resources are being
Leadership

used in the most appropriate way. The pragmatic approach argues that leaders need to
create the vision that quality matters and quality issues are worth striving for (Bate, 1998).
In essence this means that concern for quality should infuse all aspects of the organization's
work and be sustained t"rough monitoring and re-audit (Berwick, 1996).

elements.
: The ideal culture is informed by four principal


,

1. Staff are prepared to report errors or near-misses, which the organization analyses and
provides feedback on any action being taken.
2. The culture is just, and staff are able to trust the organization to distinguish acceptable

from unacceptable behaviours .
3. The culture is flexible, respecting the skills and abilities of front-line staff and allowing
them to take control.

4. The culture is prepared to learn and has the will to implement necessary major reforms .
An organization that is committed to quality improvement can

be thought of as a learning organization (Argyris, 1991). This concept distinguishes
organizations by how supportive they are to new ideas .
concentrates
: Knowledge
on how organizations
management,
become
another
more intelligent
developing
and work
area of interest,
better and more
intelligently.
how to improve
This approach demonstrates
performance
necessarily shared by the workforce.
is often
a very important
Knowledge
already present
management
principle: that tacit knowledge
in an organization, but
recognises that organizations
of
is not
need to develop a culture and structures to spread that knowledge 50 that it is useful to the

organization .
i
Summary
Improvements in care implemented

evaluated, sustained, and reinforced
as a part of clinical
within a supportive
aucit
environment
must be monitored,
.

Structures and syltems must be developed to enable organizations to integrate
improvements within a planned strategy .
A culture is required that makes the user's experience the primary motivation for

improvements,
inadequate
creates
performance,
Systems, structures,
confident staff who do not
and has clear and constant objectives .
and specific mechanisms
fear
are available
reporting
for monitoring
or confronting
sustained
improvement.
i
Acknowledgements
1. Undertaking a clinical audit project: a step-by-step guide; Published by Royal College of
Psychiatrists


Monitor and evaluate the change

Post intervention data:
maintaining the improvements
for assessing and

Process: sample
collection, and analy,is
selection, information

Modifications
interventions,
reached
of
if
the plan or additional
performance target not

Using IT

Integrated IT strategy can help data collection,
Computerised patient records (EPR/EMR) can

provide automatic and instantaneous audit
data
Individuals, team leaders, and managers -

have immediate access to current levels of
care
Data Driven Decision Making

Other tools for sustaining
improvement
Clinical performance indicators

Monitoring
Monitoring
errors
adverse incidents

Significant event audit
www,doh,gov,uk\nhsperformanceindicators


Evaluating audit quality
Checklists
Reviews
Feedback
Benchmarks
Standardized and Evidence Based Practice
Using existing frameworks
An organization may benefit from fitting the
planning and reporting of audit into an

established framework that already operates
at strategic, organisational, and clinical levels .

The European Foundation for Quality
Management (EFQM) Excellence Model
Concentrates on the whole of an
organization's activities
Is concerned with internal processes and use

of resources as well as performance and
outputs
Applies assessment processes

where an organization is performing
to indicate
well and
where poorly

The European Foundation for Quality
Management (EFQM) Excellence Model

Provides clearly defined
baseline for continuous improvement
standards as a

Provides results on the basis of which the
most effective approaches can be designed

and resources deployed
Provides means of assessing and reviewing

the effectiveness of the approaches chosen
Leadership .'
Meets Mission, Vision &Values
Leadership: vital role in QI

Guiding & Right resource allocation
Focused

Sustained QI in practice
Salford-Diabetes Clinical team
- b..!l2.;i.m~ww,med i line.ox.ac.uk Ib3ndolier I! moAn/! m p04/i4-

03.html
South Tess - colposcopy service
- b.l!Q://www.medicine.ox.acuklbandolierilmofl.ct!imp04M-
02.hlml
East Kent-Primary Care Clinical Hfectiveness
- h:J.lU/w ...w med,cine.ex.lIc ul::1bandolierllmoAet/imp01/EASTK
ENT.html
DJrest-$uccessful
Effectiveness
- htto://www
4 hlml
Cardiac Care
in Dorest (SUCCEED)
based on Evidence
med'cine.ol\.ac uk!bando!\erilmoArt!impOS/i9-
for


Some do's and don'ts ...
Careful planning
Involve others
- Especially if outcome
Open and transparent
is likely to affect them
- Never secretive
Positive and constructive
- Don't be a defeatist
Stay focused (don't get distracted)

Some do's and don'ts
Non confrontational
Non aggressive
Not over critical
Non judgemental
Don't investigate others

Way forward
When done well, clinical audit has provided a way in

which the quality of the care can be reviewed
objectively, within an approach that is supportive
and developmental.
Clinical audit provides practitioners with a systematic

response that compares the care provided to best
practice while preserving the central role of the
clinical team in agreeing and implementing plans for
change .

r----------------------------------------------- - -----

Audit criteria

http://www.nice.org.uk!usingguidance/implementati
ontools/clinicalaudit.jsp

References

Principles For Best Practice in Clinical Audit.
- National Institute of Clinical Excellence (NICE).
2002.

A Practical Handbook for Clinical Audit_
- Published in 2005 Endorsed by the Healthcare
Commission, National Institute for Clinical
Excellence (NICE), the Clinical Governance Support
Team (CGsT) and the National Audit and
Governance Group (NAGG).

References
Undertaking a clinical audit project:
guide; Published by Royal College of Psychiatrists
a step.by.step

"How
Published
To"
by
guide on Clinical Audit (Version
UH Bristol (University Hospitals, Bristol)
3);


NOTES
.

'
",
<{::>.

,0.'
.

.
'
'
.


NOTES


Department
Case Study 1
of Cardiology

To improve
contraindication.
the
with acute M.1. potential

use of beta blockers in patients
Beta blockers are an essential component
with acute Mol. without"any
of the management
beneficial effects of beta blockers in this setting include .
obvious
of patients
a) Decreased O2 demand and relief of ischemic chest pain due to the acceptable
in heart rate, BP and contractility .
reductions
b) Decreased
demonstrating
risk
showing a reduction
of ventricular fibrillation
an increase in the ventricular
in sudden cardiac death .
as suggested
fibrillation
by experimental
threshold
studies
and by clinical trials
c) Decreased automatically,
slowing of conduction .
increased electro-physiologic threshold for activation and

d) Bradycardia prolongs diastole and therefore improves coronary diastolic perfusion and
reduces after de-polarisation and triggered activity .
e) Reduction in remodelling improvement in left ventricular haemodynamic function,
f)
depending
Improved
upon infarct size and the timing oftreatment
left ventricular diastolic function

.
with a less restrictive filing pattern .
Objectives:
1) To ensure that beta blockers were given to patients

any absolute contraindication .
with acute M.1. in the absence of

2) To ensure that beta blockers were administered early enough after presentation i.e .
within 12 hours as recommended in major studies .


Standards:

1) At least 80% of patients
given beta blockers.
with acute M.1. and without any contraindication should be

2) In at-least 80% cases beta blockers should have been administered
presentation.
within 12 hours of

The files of patients with acute MI admitted in our CCU in the last three months were

evaluated (July 2005- September 2005).

Total Number of patients evaluated was 27.
1. Twelve patients had absolute contraindication to beta blockers as follows:
a.
b.
Five patients had advanced AV blocks
Three patients had significant sinus bradycardia

c. Three patients had cardiogenic shock

d.
2. Thirteen
One patient had significant hypotension
patients were given beta blockers.

requiring inotropic support

a.
b.
Of the thirteen
hours
Six patients
patients
received
seven patients
beta blockers after twelve

received beta blockers within
hours but within

twelve
seventy two

hours as follows:-
i. Two patients after settling their lVF
ii. Three patients

anticipating
caution
Brady-arrhythmias
taken in view of acute inferior wall M.1.

3.
iii. The c"use for the delay in the 5
Two patients without any obvious contraindication

th
patient was not clear from the data
did not get beta blockers.


. /

86% of patients
them within
who deserved beta blockers received them and 54% of patients
twelve hours. 46% of patients received them beyond twelve hours but within
received
seventy two hours. However 83% of those who received beyond twelve
seventy two hours had valid reason for the delay .
hours but within
A more liberal though cautions use of beta blockers in acute inferior M.1. if the heart rate is
acceptable, should be encouraged as it was noted in case of inferior M.1.
Literature and data search is in favour of more liberal and early use of beta blockers in the
settling of acute M.1. (both anterior

with practicing
protocols,
and inferior)
clinicians and cardiologists through
and sharing the information
CME's, Clinical meetings,
holding classes and discussing with junior doctors .
and learning
updating CCU


51.
No. No.
13
(

1.
2.
70447
70483
Anterior MI
Acute extensive M I
No
No
Sinus Bradycardia
' Cardiogenic
with CHB
Shock
3. 70673 Acute extensive MI No Cardiogenic
with CHB
Shock

4. 70693 Acute inferior MI No
6

5. 70760 Acute inferior MI Yes
6. 70767 Anterior MI Yes 2
7. 70778 Anterior MI Yes 72 After settling LVF
8. 70811 Non QMI Yes 9
9.
10.
11.
70846
70847
70855
Anterior
Anterior
Inferior
MI
MI
rill
No
No
No
I Hypotension
Sinus Bradycardia

12. 70855 Anterior & Inferior Yes 4
13. 70880
MI
Inferior rill No Hypotension,
Bradycardia
Sinus
14.
15.
71006
71047
Inferior Lateral MI
Anterior MI
Yes
Yes 12

16. 71066 Anterior MI No
17. 71045 Inferior MI No Complete Heart Block
18. 71123 Anterior MI Yes 72 After controlling LVF
19.
20.
71275
71315
Inferior MI
Inferior MI
Yes
No
,
I
,I
4
Complete Heart Block,
Cardiogenic Shock

21.
22.
23.
71372
71385
71345
Inferior
Inferior
Inferior
MI
MI
MI
No
Yes
No
I

24.
25.
71415
71417
Anterior
Anterior
MI
MI
Yes
Yes
0.5
28

26. 71514 Inferior MI No Complete Heart Block
27. 71521 Inferior MI Yes 72


MedicaLAudit - Cardiology

Compliance with betablocker therapy onthe prescription at the time of discharg
1%
El Patients who were on betablockers
51Patients who had bradycardia contraindicating

betablockers
o Patients who had CGPOand hence no~ put on

betablockers
o Betablocker therapy not prescribed
Event audited: Whether patients with angiographically

after
PTCAare prescribed beta-blockertberapy7
I.:: - -
proven coronary artery disease and patients
-----------------------------]

I Inference: Hence there was a overall 98% compliance with beta blocker therapy on the prescription _
i at the time of discharge.
'---- - - - -.------------

--I

Case Study 2
Department of Urology

CaseStudy 2

Topic: TURP

Purpose: To improve patient satisfaction
after TURP

Background ...
Unless selected carefully.

up to 30% of patients can be dissatisfied
with the result ofTURP

Background ...

Causes of urinary difficulty in elderly men:
- Obstructed urination (prostate)

- Overactive Bladder
- Combination of above two conditions
- Underactive 6ladder

TURP is excellent operation for obstruction but
ineffective if main proble,,, is bladder over- or under-
activity.


Background
Patients with most severe symptoms are most likely

to show maximum imprJvement after surgery, while
those with only mild/moderate symptoms may not
have much improvement .

Symptoms may remain unchanged, and sometimes
worsen after TURP, if underlying problem is anything
other than obstruction.
IPSSis best way of measuring severity of symptoms

and Uroflowmetry (U/FI is the best way to screen
objectively for bladder o:Jtlet obstruction .

Case Study 2: Objectives
1. Improve patient selection for surgery by using IPSScut-
off of >20/35 and
2. Peak Urine Flow rate (~l-'<) cut off of <15ml/sec .

Case Study 2: Standards
1. 100% of patients being considered for TURP should

have IPSSrecord and Lroflowmetry before surgery
(excluding those already in urinary retention) .
2. 90% of all operated patients should be reviewed 6

weeks after surgery, vvith repeat IPSS and V/F, to
assess patient satisfaction with surgery .
3. 90% of patients should have IPSS <8/35 and Omax>

18ml/sec after TURP.

Case Study 2: Data collection
Pre-op (n=77)

itMcm$i
<15 >15

Urinary Difficulty
Haematuf,la -
Bladder Stone
Faih:!d:Me-dic~i-f;:~~tment-:
56
2
2
7
66
7
2
1

Case Study 2: Data Collection
Post-or (n=66)

Indication thr SUI'1Eq
Urinary difficulty 7 56 3 59
Haematuria 2
Bladder Stone
Failed Medical trea!",ent 2

2
3 2
2
4 1

Case Study 2: Results
66/77 (86%) patients returned for 6-week review
61/66 (92.4%) had acceptable improvement in

symptoms
65/66 (98.5%)
urinary flow rate
had acceptable improvement in

76% success rate before instituting
Uroflowmetry based screening
IPSS and

Page 112 of 117

Case Study 2: 6 weeks follow up
100%
90%
HO%
70%
86%
Target = 90%
60%
soy,
'0%
30%
20% 14%

10%
0%
Done Not done
[1Done g Not done

Case Study 2: IPSS
7.60% Target = 90%
.-- < 8

~>8

Case Study 2: IPSS

7.60% Target =90%
!,~ff~tjr~i
..
,'< .. ::- < 8
:", > 8

------------------------------------------=- ~

Case Study 2: Qmax > 18ml/sec
2.50%. . Target;; 90%

I: > 18 ml/see < 18 ml/sec

Importance

76% success rate before instituting IPSS
and Urofiowmetry based screening

Case Study 2: Opportunity
Improvement
for

,/ Formal urodynamics should be considered, before
proceeding with surgery,

r f.Jr men
;,;..
who fail medical treatment
to try and identify those with detrusor under-activity,
whose symptoms and flow rate may not show adequate
improvement after surger','


Exercise 6
Kindly develop a clinical audit for any medical/surgical

incorporating
topic of your choice
all that you have learnt in this workshop and case studies 1
and 2.

Interactive workshop on c1ioicalaudit Page 115 of 117

interactive workshop on clinical audit Page 116 of 117

Interactive Workshop On Clinicla Audit

Загружено:

Сведения о документе

Оригинальное название

Авторское право

Доступные форматы

Поделиться этим документом

Поделиться или встроить документ

Параметры публикации

Этот документ был вам полезен?

Это неприемлемый материал?

Авторское право:

Доступные форматы

Interactive Workshop On Clinicla Audit

Загружено:

Авторское право:

Доступные форматы

'

2nd Floor, Institution of Engineers Building

Telefax: 91-11-2337 9321, 2337 9621

Web site: www.gcin.orgj http://nabh.co/main/

Stage 1: Preparing for audit and selection of topic

6. Stage 2: Selecting criteria 39

7. Stage 3: Measuring level of performance 57

8. Stage 4: Making improvements 76

9. Stage 5: Sustaining improvement 94

Interactive workshop on clinical audit Page 2 of 117

1. The participants are expected to be punctual for the sessions,

., 3, The use of cell/mobile

vibration) mode so as not to cause disturbance to fellow participants. The

under emergency situation .

5. The participants are expected to actively participate in all the sessions so

6. Grievance, if any, shall be communicated

3. To use evidence of good clinical practice/evidence based medicine in

4. To help clinicians formulate measures of quality based on evidence of good

08:30 a.m. to 09:00 a.m.

09:00 a.m. to 09:30 a.m .

09:30 a.m. to 10:00 a.m . Clinical audit - an overview

10:00 a.m. to 11:00 a.m .

11:00 a.m. to 11:15 a.m .

11:15 a.m. to 12:30 p.m .

Stage 2: Selecting criteria

12:30 p.m. to 01:15 p.m.

01:15 p.m. to 02:00 p.m. LUNCH

02:00 p.m. to 02:30 p.m.

03:30 p.m. to 03:45 p.m. TEA BREAK

03:45 p.m. to 04:45 p.m . Case study

Wrap up; Sharing of best practices;

04:45 p.m. to 05:00 p.m .

Interactive workshop on clinical audit Page 5 of 117

What is Clinical Audit?

Principles for Best Practice in Clinical Audit (2002, NICE/CHI)

The key messages being that:

All parties, those being audited

The History of Clinical Audit

in the 1989 White Paper

Clinical audit is gaining popularity

Interactive workshop on clinical audit Page 7 of 117

10. Analysing the re-audit data.

What Clinical Audit Is Not?

Other forms of audit can include:

view of the organization's financial position at a given time .

Interactive workshop on clinical audit Page 9 of 117

being met. Results are specific and local to a particular

A clinical audit project will never involve a completely

" patients, service users or carers at a number of levels, e.g .

Involvement in the design of individual clinical aUdii~TOjects or whole programmes

I May involve experiments based on a Never involves experiments and involves

It may involve random allocation .

It is a systematic review of practice.

There may be extra disturbance to patients . There is little disturbance to patients.

It could be a new treatment. It never involves a completely new

May involve experiments on patients . Patients continue to experience their normal

'. Interactive workshop on clinical audit Page 11 of 117

Extensive statistical analysis of data is Some statistics may be useful. '.

Responsibility to act on findings is unclear.

Responsibility to act on findings rests with

Always requires ethical approval.

Research can identify areas for audit.

Audit can be a precursor to clinical research