MANAGEMENT NURSING ASSIGNMENT

CITRANINGRUM PUTRI KURNIA

P1337420617058

Tittle/ye Research Aim Design Result Strength Weakne Implic

ar er ss
The Dawn to develop a Mixed Outputs from This This Sampl
Nursing Connolly nursing method, the NQI journal journal was lim
Quality and Fiona quality including framework focus on just to 53 E
Indicator Wright indicator patient domains offer just one based on speaki
Framewo (NQI) records a more tool, and framewo patient
rk Tool framework audit, comprehensiv explain rk tool, consen
(2017) and provide patient e very so we partici
a experience understanding clear don’t in the
comprehensi questionnai of nursing what is know
ve reporting re, nurse quality the what
mechanism self-report compared to meaning patients
for nursing questionnai when domains and the feel if
care. re and are analysed result there is
collecting separately. from no
ward-level The NQI framewo indicator
informatio framework rk tool. in
n. The also provides framewo
sample was a more rk tool.
53 patients inclusive
and 22 mechanism
nurses. for assuring
nursing care.

Identifyi - Efi To identify This is an The review This The QIs are
ng Evangelo QIs integrative identified 13 research indicator screen
Validated u RN; associated review of studies and 45 focus on s don’t tools, w
Nursing BSc, with nursing published QIs associated ICU describe identif
Quality MSc care for literature with nursing ward, so clearly, potent
Indicator - adult ICU in based on a care in the we can they subopt
s For The Ekaterini the modified ICU. The set know used clinica
Intensive Lambrino literature. framework of QIs specifica theory as and rev
Care u RN; The for review assessed in lly the indicator specifi
Unit: An BSc, methodologi research each kind of s. So, proble
Integrativ MSc, cal quality (Whittemor study, the patients not areas t
e Review PhD of QIs was e and Knafl type of nurse they used everyon need fu
(2018) - assessed and 2005). The staffing for e would investi
Christian associated process for measure as research, understa (De Vo
a Kouta variables of data well as because nd al., 200
RN; BSc, quality and extraction morbidity in the what’s QIs ma
MSc, quantity of and and mortality ICU their classif
PhD nursing care synthesis rates varied patients research. accord
- Nicos were also was based considerably. need three
Middleto identified. on a Findings nursing dimen
n BSc, detailed suggest that cares which
MSc, pre- quality and more corresp
PhD specified quantity of than the to
 protocol, as nursing care other structu
presented are strongly wards. proces
in the associated outcom
sections to with higher compo
follow. rates of of qua
adverse (Dona
events, ,
mortality, 1992).
infections and
complications
in adult ICUs.
Methodologic
al quality of
the
QIs also
differed
considerably.
Higher AIRE
scores,
indicating
higher
scientific
evidence of
methodologic
al quality, can
be
used to select
evidence-
based and
valid QIs.
Patient - Anita To evaluate A cross- Patient s were This They The ne
Satisfacti Karaca patients’ sectional, more satisfied journal used 653 increas
on With - Zehra satisfaction descriptive with the has many patients quality
The Durna with the survey “Concern and indicator who healthc
Quality quality of study. Caring by s and discharg
Of nursing care Nurses” and many ed from service
Nursing and examine less patients hospital been
Care associated satisfied with from and that identif
(2017) factors. the different was health
“Information age, job, really related
You Were gender, huge inform
Given.” etc to amount. and
Patients gather So, it advanc
(63.9%) data. So needs techno
described we know long change
nursing exactly time to expect
care offered what gather and op
patients the data about
during feel and since the health
hospitalizatio want intervie an incr
n as when we w was in
excellent.Pati as nurses face to individ
ents who were give face. involv
18-35 years nursing in thei
old, care. health
And they care an
married, explain increas
college or very cost an
university clearly in compe
graduates, each ess in
treated at the indicator health
surgery and s. (Freita
obstetrics- Silva,
gynae- Minam
cology units, Bezerr
and patients &Sous
who stated 4).
their health as
excellent and
hospitalized
once or at
least five
times were
more satisfied
with the
nursing care.
According to
this study, the
nurses needed
to show
greater
amount of
interest to the
information-
giving
process.
 From all of those journals above, I can conclude that all of them has the similar implications. There
is quality of healthcare or nursing care for patients. Patient satisfaction is a concrete criterion for
evaluation of health care and therefore quality of nursing care. The quality and adequacy of healthcare
services can be measured based on views and satisfaction of patients and their relatives. The surveys in
health services concerning health satisfaction are carried out to evaluate the patient satisfaction, to learn
patient’s expectations, their suggestions and feedbacks, make the quality improvement constantly in all
service periods, to search the effects of socio-demographic and treatment periods on patient satisfaction.
And I conclude from three journals that patients would really happy if the nurses can taking care of them
and have good skill in communication. So I suggest for the better in the future, nurses have good skill in
communication, especially terapeuthic communication. Hope we can explain the patients need clearly and
make them comfort with us.
Recei ved: 27 October 2017 Revised: 3 November 2018 Accepted: 3 December 2018

DOI: 10.1002/nop2.237

RE SE ARC H ARTI C L E

Patient satisfaction with the quality of nursing care*

Anita Karaca | Z ehra Durna

Florence Nightingale Hospital School of

Nursing, Istanbul Bilim University, Istanbul, Abstract
Turkey Aim: To evaluate patients’ satisfaction with the quality of nursing care and examine

Correspondence associated factors.

Anita Karaca, Florence Nightingale Hospital
School of Nursing, Istanbul Bilim University,
Istanbul, Turkey.
Email: anitakaraca@hotmail.com
Design: A cross-sectional, descriptive survey study.
Methods: The sample was composed of 635 patients discharged from a private hos-
pital. Data were collected using “Patient Satisfaction with Nursing Care Quality
Questionnaire” with a total of 19 items, and a questionnaire designed to record so cio-
demographic characteristics and medical histories between January 1-May 31, 2015.
Results: Patients were more satisfied with the “ Concern and C aring by Nurses” and less
satisfied with the “Information You Were Given.” Patients (63.9%) described nursing
care offered during hospitalization as excellent. Patients who were 18-35 years old,
married, college or university graduates, treated at the surgery and obstetrics-gynae-
cology units, and patients who stated their health as excellent and hospitalized once or
at least five times were more satisfied with the nursing care. According to this study, the
nurses needed to show greater amount of interest to the information-giving process.

K EY WORD S

nursing care, patient satisfaction, patients’ expectations, quality of car e

1 | INTRODUCTION (Merkouris et al., 2013). Pati ent satisfaction is the most im p or t-

Increasing competition in every field today also affects the health-
care industry. The most important competitive advantage of health
service providers is to provide quality health services (Alsaqri, 2016;
Reck, 2013; Şişe, 2013). The need for increased quality of healthcare
services has been identified via health-related information and ad-
vances in technology, changes in expectations and opinions about
health care, an increase in individuals’ involvement in their health
care and increased cost and competitiveness in the health sector
(Freitas, Silv a, M inam isav a, Bezerra, & Sousa, 2014).
The quality and adequacy of healthcare services can be m ea-
sured based on views and satisfaction of patients andtheir relatives
*C orrecti on a d d e d on 22 Janu ar y 2019, after first online publication: a typographical error in
the article title has been corrected.
This article was originally written as a doctoral thesis at Istanbul BilimUniversity Institute of
Health Sciences Nursing PhD Program
ant indicator of quality of care and it considered an outcome of
healthcare ser vices (Abdel Maqsood, Oweis, & Hansa, 2012;
Akhtari-Zavare, Abdullah, Syed Hassan, Binti Said, & Kamali, 2010;
Mohanan, Kaur, Das, & Bhalla, 2010). Patient satisfaction measure-
ment provided crucial information on performance thus contribut-
ing to total quality management (Goh, Ang, Chan, He, & Vehvilainen
Julkunen, 2016; Shinde & K ap u r k ar, 2014). Total quality manage-
ment includes professional knowledge, competence and application
of appropriate technolog y, the p atients’ percep tion about the type
and level of the care they have received (Özsoy et al., 2007; You
et al., 2013). In today ’s consu m er-oriented healthcare markets, a
patient-centred measure of satisfaction with the quality of nursing
care received is a major component of hospital quality management
systems (Laschinger, Hall, Pedersen, & Almost, 2005). Patients need
their problems diagnosed and treated properly, their function re-
stored and/or symptoms relieved. If the results are unsatisfactor y,

This is an open access article under the terms of the Creative Commons Attribution License, whichpermits use, distribution and reproduction in any medium,
provided the original work is properly cited.
© 2019 The Authors. Nursing Open published by John Wiley &Sons Ltd.

Nursing Open. 2019;6:535-545. wileyonlinelibrary.com/journal/nop2 535

KARACAAn d d UR nA
consumers will change the healthcare facility they applied for
treatment and care (Ksykiewicz-Dorota, Sierpińska, Gorczyca, &
Rogala-Pawelczyk, 2011; Shinde & Kapurkar, 2014). Patients who
are more satisfied with their care are more likely to follow medically
prescribed regimens and thus contributing to the positive influence
on health (Buchanan, Dawkins, & Lindo, 2015; Dzomeku, Atinga,
Tulukuu, &Mantey, 2013; Fröjd, Swenne, Rubertsson, Gunningberg,
& Wadensten, 2011). More satisfied patients are more liable to rec-
ommend the hospital to family and friends (Buchanan et al., 2015;
Mohanan et al., 2010). Patients’ opinions are the best source that can
tell the providers of what is important, that is why this information
can be used in healthcare planning and evaluation (Abdel Maqsood
et al., 2012; A lsaqri, 2016; M erkouris et al., 2013; V illarruz-Sulit,
Dans, & Javelosa, 2009). All these changes and developments in
the healthcare field require restructuring of all healthcare services,
including nursing, through questioning the quality of treatment ser-
vices (Şişe, 2013).
1.1 | Background
Patient satisfaction is a concrete criterion for evaluation of health
care and therefore quality of nursing care (Alhusban & Abualrub,
2009; Shinde & K ap urk ar, 2014). It provides crucial inform ation for
healthcare managers by providing im p or tant resources for processes
such as those involved in measuring patients’ expectations and sat-
isfaction with nursing care quality, improving nursing service quality
through identification of areas of failure and planning and imple-
menting necessary training (Abdel Maqsood et al., 2012; Gadalean &
Cheptea, 2011; Geçkil, Dündar, & Şahin, 2008). Evaluation of health care
involves defining the objectives of care, monitoring healthcare inputs,
measuring the extent to which the expected outcomes have been
achieved and assessing the extent of any unintended or harm-
ful consequences of the intervention (Alsaqri, 2016; Sitzia & Wood,
1997; Tang, Soong, & L im , 2013).
Nursing care is one of the major components of he althcare ser-
vices (Buchanan et al., 2015; M erkouris et al, 2013; Mohanan et
al., 2010; Sitzia & Wood, 1997). Patients’ sati sfaction with nursi ng
care has become an established as the most important predictor
of the overall sati sfaction with hospital care and an im p ortant goal
of any healthcare org anization (G oh et al., 2016; L aschinger et al .,
2005; Mohanan et al., 2010; Reck, 2013). M easuring patients’ sat-
isfaction with nursing care could be effective in improving nurs-
ing service quality by facilitating the creation of standards for care
while monitoring both results and patients’ perceptions of quality
(Akın & Erdoğan, 2007; Senarath & Gunawardena, 2011; Tang et
al., 2013). The nurses have a central role in offering em otional and
psychological support to patients and their families in all settings,
such as supporting the patient through diagnosis and ensuring op-
timum care given to them. Besides the provision of technical care,
nurses must have the qualified professional knowledge, attitudes
and skills, providing the informational, emotional and practical sup-
ports (Akhtari-Zavare et al., 2010; Buchanan et al., 2015; Goh et al.,
2016).
536
If healthcare organization managers are able to identify patient
exp ectati ons, they could accordingly adjust the performance of
ser v i ces that they offer to meet these exp ectati ons ( Frei tas et al ,
2014; Fröjd et al., 2011; M ilutinovic, Sim in, Brkic, & Brkic, 2012).
The sur veys in health ser vices concerning health satisfaction are
carried out to evaluate the patient satisfaction, to learn patient’s
exp ectati ons, their suggestions and feedbacks, make the qual-
ity improvement constantly in all service periods, to search the
effects of socio-demographic and treatment periods on patient
satisfaction (Buchanan et al., 2015; Özer & Çakıl, 2007; Sitzia &
Wood, 1997). That is why patient satisfaction should be measured
constantl y u sin g v alid , r elia b le assessment instruments to assess
care quality, identify variables that affect care and determine
which items should be prioritized and which require alteration in
the ser vice based on patients’ responses (Buchanan et al., 2015;
Merkouris et al., 2013). A good assessment instrument measur-
ing the factors that determine patient satisfaction should be de-
veloped to improve nursing service quality (Freitas et al., 2014;
Laschinger et al., 2005). Therefore, the findings of nursing man-
agement research should be used as an indicator of the contribu-
tion made by nursing to the patient care process and this could
aid the advancement of the profession in terms of scientification
(Alsaqri, 2016; Freitas et al., 2014; Goh et al., 2016).
1.2 | Resear ch questions
• What is the satisfaction level of patients about the quality of nurs-
ing care?
• Is there any relationship between patients’ satisfaction with the
quality of nursing care according to their socio-demographic char-
acteristics and medical history?
2 | METHODS
2.1 | Design
The study used a descriptive, cross-sectional research design.
2.2 | S etti ng and sampl es
Participants included 635 hospitalized patients receiving internal
medicine, surgery and obstetrics and gynaecology services at a pri-
vate hospital between January 1-May 1, 2015. The sampling criteria
were as follows: patients aged 18 years or older, patients who were
discharged, hospitalized for at least 2 nights at the time of data col-
lection, able to speak and understand Turk ish , not too confused or
ill to complete the questionnaires and agreeing to participate in the
study.
The response rate of this study is 92.8%. The survey was not ad-
ministered to all patients who had not planned their discharge (those
were decided or wished to be discharged suddenly) or were trans-
ferred to another hospital. Incompletely filled out surveys were not
included in the study.
KARACAAn d d UR nA
2.3 | Ethical considerations
Prior to data collection, the research protocol was reviewed
and approved by the relevant scientific ethics committee (IBU
Clinical Research Ethical Committee, Ethical Approval Number:
01.11.2014/25-168). Permission to conduct the research was also
obtained from hospital administrators. Written approval to use
the Patient Satisfaction with Nursing Care Quality Questionnaire
(PSNCQQ) and translate it into Turkish was obtained from
Laschinger, who developed the scale. All patients provided written
informed consent.
2.4 | M easurem ent
low, moderate, high and very high. Perceived health was measured
by a self-repor ted question was graded by six variables prior to their
2.4.1 | P a t i e n t Sa t i s fa c t i o n with Nursing Care
Quality Questionnaire
The PSNCQQ was designed to measure the extent of anticipated
need, assess patient satisfaction following short-stay hospitaliza-
tion and determine the influence of socio-demographic, personal
and other factors at a minimum level. The scale was developed
using the Patient Judgements of Hospital Quality Questionnaire,
which was developed by a multidisciplinary research team at the
Hospital Corporation of America (Laschinger et al., 2005; Reck,
2013).
The scale consists of 19 items pertaining to features of a wide
range of nursing activities including n urses’ attention, kindness, re-
spect, courtesy, skills, competence and fulfilment of patient needs.
As it is short and it can be completed easily, it has verygood psycho-
metric properties that can be used by managers in quality improve-
ment activities (Fröjd et al., 2011; Laschinger et al., 2005). Each item
consists of a “ signpost,” which is a phrase designating its content and
a “descriptor,” which is a detailed question. The scale also includes a
general p erceptions section consisting of four additional questions
designed to measure satisfaction with the overall quality of care
and treatment received during hospitalization, the overall quality
of n u rsin g care, thoughts on overall heal th and the likelihood that
the patient would recommend the hospital to relatives and friends
(Laschinger et al., 2005; Milutinovic et al., 2012).
The scale was designed for application by administrators in areas
requiring improvement, to provide patient-oriented outcomes and
2 .7
for the identification of strong and weak aspects of the nursing care
process. Items were based on factors identified as important ele-
ments of patient satisfaction with nursing care. The PSNCQQ can
be incorporated into existing hospital quality monitoring systems to
monitor patient satisfaction. In addition, the PSNCQQ can be used
as an evidence-based indicator given its contribution to the patient
care process as a result variable, to evaluate changes in depart-
mental and institutional processes. This feedback provides useful
information to nurse administrators (Abdel Maqsood et al., 2012;
Laschinger et al., 2005).
Participants’ responses are provided using a 5-point Likert-type
scale. Total possible scores range from 19-95. Lower total scores
537
indicate greater satisfaction with nursing care. The scoring of the
scale was: 1 = excellent, 2 = very good, 3 = good, 4 = fair, 5 = poor.
2. 5 | D at a collection
Data were collected using the PSNCQQ, which measures health-
related prop er ties considered to affect pati ent satisfaction and a
questionnaire, designed in the light of related literature, to record
socio-demographic characteristics and medical history. A question-
naire consisting of 16 items pertaining to variables affecting pa-
tient satisfaction was developed according to these characteristics.
Income levels were measured by the patients’ self-perception of
their economic status and lifestyle. It was presented in four options:
admission as excellent, good, fair, poor, very poor and unsure.
The data were collected by the researcher. The patients com-
pleted the questionnaires prior to their discharge from the hospital.
The patients who agreed to participate in the study were provided
with an explanation about the purpose of the study and they signed
informed consent forms. Those who refused to participate reported
that they did not have the time or were just not interested in partici-
pating. Data were collected by face to face interviews from illiterate
patients.
2. 6 | D at a a n al y s is
Data were analysed using SPSS software (IBM Corp. Released
2012. IBM SPSS Statistics for W indows, Version 21.0; IBM Corp,
Armonk, NY, USA). The analysis included descriptive statistics
such as frequencies, means, standard deviations and percentages.
The distribution of the data was assessed using the Single Sample
Kolmogorov-Smirnov test and as the significance values exceeded
0.05, parametric tests were used in the advanced-level analysis.
About the parametric tests, t tests were performed to analyse in-
dependent variables with two categories, one-way ANOVAs were
performed to analyse independent variables with more than two
categories and Pearson’s correlation coefficients were used to ana-
lyse relationships.
| Validity andreliability analysis
The PSNCQQ was translated into Turkish and the linguistic, and
concep tu al equ i v al ence of the items was establ i shed . Back tr ans-
lation was performed to ensure language equivalence between
the English and Turkish versions of the scale. The original scale
was translated into Turkish linguists who were highly competent
in both languages. Five bilingual experts consisted of a doctor,
two nursing faculty members, a nurse manager and a linguist.
Expressions used in the scale were analysed individually and in
combination and optimal expressions were selected by forming a
pool of 19 items. Back translation from Turkish to English was per-
formed by two trained linguists (English teachers) with knowledge
KARACAAn d d UR nA 538

and experience in both languages. The back-translated and original

versions of the PSNCQQ were compared and found to be highly TABLE 1 Patient characteristics (N = 635)
similar in meaning and reorganized based on the ch aracteristics
Variables N %
of the countr y. After then, the expert met and reviewed to deter-
Age(years)
mine the scope of the validity of the scale. Eight experts (nurs-
18-35 239 37.6
ing academicians specialized in medical nursing, surgical nursing
and nursing administration) provided opinions about meaning and 36-55 180 28.3
content sufficiency. A pilot study was then conducted from 1-31 56 andmore 216 34.0
December 2014 to determine whether there were any unclear Gender
questions in the scale. The data from the pilot study were then Female 491 77.3
excluded from the final data analysis. According to the results of Male 144 22.7
the pilot study, small changes were then made to the expressions Marital status
in some scale items to increase their understandability.
Married 473 74.5
In this study, the coefficients for correlations between average
Single 65 10.2
PSNCQQ item scores ranged from 0.80-0.89, which demonstrated
Divorced 22 3.5
an appropriate level of reliability. Cronbach’s α for the PSNCQQ,
Widowed 75 11.8
calculated to determine internal consistency and uniformity, was
Education
0.98, which was very h ig h . In L a sch in ge r et al .’s (2005) study, the
coeffi ci ents for the correl ati ons between PSNCQQ items ranged Illiterate 14 2.2

from 0.61-0.89 and were described as high and Cronbach α was Literate 16 2.5
0.97, which was described as excel l ent. Therefore, the results Primary school 114 18.0
obtained in the current study were similar to those reported by Secondary school 54 8.5
Laschinger et al. (2005). In view of this, the Turkish version of the High school 200 31.5
PSNCQQ could be considered to possess excellent psychometric College or University 211 33.2
properties, which were similar to those reported for the o riginal Postgraduate 26 4.1
scale.
Perceived income leve l

Very high 19 3.0

3 | RESULTS High 268 42.2
Moderate 331 52.1
Low 17 2.7
3.1 | S oci o-dem ograp hi c characteri stics and medi cal Occu pation
history
Worker (blue collars) 47 7.4
The mean age of the sample age was 47.94 (SD 19.66) years and Civil servant 47 7.4
37.6% were aged between 18-35 years. The most of par ticipants Retired 109 17.2
were women (77.3%), married (74.5%) and college or univer sity Self-emp loyment 73 11.5
graduates (33.2%), at moderate-income level (52.1%) and house-
Housewife 199 31.3
wives (31.3%). Of the patients, 2.2% were illiterate. More than half of
the sam ple (61.6%) were admitted to the ser vice directly from the patient
Student 17 2.7
Others 143 22.5
admission department and had been hospitalized once in the preceding 2
years (66.6%). The average duration of the current hos-
pitalization was (4.38SD 5.75) days(Table 1). Analysis of PSNCQQ scores for perception-related items showed
that 61.4% and 63.9% of participants’ responses for the “Quality of the
care and service provided during your stay at the hospital” and “Quality
3.2 | PSNCQQ scores
of the nursing care provided during your stay at the hospital” item s,
Analysis of PSNCQQ scores revealed that the item for which satis- respectively, were “ excellent.” In addition, 87.9% of patients stated that
faction levels were highest (1.38 SD 0.66) was the “ Concern and Caring they would recommend the hospital to their family and friends.
by Nurses: Courtesy and respect you were given; friendliness and kind-
ness” item. The item for which satisfaction levels were lowest (1.74 SD
3.3 | Comparison of PSNCQQ scores according to
0.86) was the “ Inform ation You Were Given: How clear and com plete
patients’ socio-demographic characteristics
the nurses’ explanations were about tests, treatments and what to ex-
pect” item. Overall, patients’ PSNCQQ scores ranged between 1-4.05, The mean PSNCQQ score of patients at the age of 56 years or
with an average score of 1.61 (SD 0.65). This indicated that the level of older was significantly higher (1.75 SD 0.68) in relation to those
satisfaction w ith nursing care was high (Table 2). observed for patients aged 18-35 years (1.50 SD 0.61) and aged
KARACAAn d d UR nA 539

TABLE 2 Distribution of Patient

I tems M SD Min Max
Satisfaction with Nursing Care Quality
Questionnaire (PSNCQQ) Scores (N = 635) 1. Information You Were Given: How clear and 1.74 0.86 1 5
complete the nurses’ explanations were about
tests, treatments and what to expect
2. Instructions: How well nurses explained how to 1.72 0.84 1 5
prepare for tests andoperations
3. Ease of Getting Information: Willingness of 1.57 0.76 1 5
nurses to answer yourquestions
4. Information Given by Nurses:How well nurses 1.59 0.74 1 5
communicated with patients, families, and
doctors
5. Informing Family or Friends: How well the 1.72 0.82 1 5
nurses kept them informed about your condition
and needs
6. Involving Family or Friends in YourCare: How 1.71 0.77 1 5
much they were allowed to help in yourcare

7. Concern and C aring by N ur ses: Cour tesy and 1.38 0.66 1 5

respect you were given; friendliness and kindness
8. Attention of Nursesto YourCondition: How 1.51 0.72 1 4
often nurses checked on you and how well they
kept track of how you were doing
9. Recognition of Your O pinions: How much nurses 1.74 0.85 1 5
ask you what you think is important and give you
choices
10. Consi d er ation of You r N eeds: Wil l ingness of 1.57 0.74 1 5
the nurses to be flexible in meeting yourneeds
11. The Dai l y Routi ne of the N urses: H ow wel l 1.65 0.77 1 4
they adjusted their schedules to your needs
12. Helpfulness: Ability of the nurses to make you 1.49 0.70 1 4
comfor tabl e and reassure you
13. N ursing Staff Response to Your Cal ls: H ow 1.48 0.68 1 4
quick they were to help

14. Skill and Competence of Nurses: How well 1.58 0.79 1 5

things were done, like giving medicine and
handling IVs

15. Coordination of Care: The teamwork between 1.58 0.72 1 5

nurses and other hospital staff who took care of
you
16. Restful A tm osphere Prov i d ed by N u rses: 1.60 0.81 1 5
Amount of peace andquiet
17. Privacy: Provisions for your privacy by nurses 1.53 0.73 1 5
18. Discharge Instructions: How clearly and 1.63 0.77 1 5
completel y the nurses told you what to do and
what to expect when you left the hospital
19. Coordination of Care After Discharge: N urses’ 1.7 0.81 1 5
efforts to provide for yourneeds after you left
the hospital.
Average PSNCQQ Score 1.61 0.65 1 4.05

36-55 years (1.56 SD 0.62; p < 0.001). No statistically significant
differences were found in the PSNCQQ scores of patients by gen-
der or occupation (p > 0.05).
The widowed patients’ mean PSNCQQ score was found to be
statistically higher (1.81 SD 0.75) than that of the married patients’
(1.57 S D 0.62), and the di fference was si g ni fi cant ( p < 0.05). T he
patients’ PSNCQQ scores varied significantly by education level
and income (p < 0.001). The literate patients and patients with
moderate incomes scores were higher (2.02 SD 0.65, 1.71 SD 0.68,
respectively) than those of the patients who had completed col-
lege or universit y and p ati ents w ith h ig h i ncom es (1.52 SD 0.60,
1.48 SD 0.56, respectively; Table 3).
KARACAAn d d UR nA 540

TABLE 3 Comparison of Patient Satisfaction with Nursing Care Quality Questionnaire scores based on patients’ socio-demographic
characteristics (N = 635)

Statistical eval u ation

N M ± SD t/F p

Age(years)
***
18-35 239 1.50 ± 0.61 F: 9.506 <0.001
36-55 180 1.56 ± 0.62
56 and m ore 216 1.75 ± 0.68
Gender
Female 491 1.59 ± 0.65 t: 0.102 0.317
Male 144 1.65 ± 0.63
Marital status
*
Married 473 1.57 ± 0.62 F: 3.234 0.022
Single 65 1.64 ± 0.73
Divorced 22 1.57 ± 0.55
Widowed 75 1.81 ± 0.75
Educational background
***
Illi terate 14 2.00 ± 0.77 F: 3.042 0.006
Literate 16 2.02 ± 0.65
Primary school 114 1.65 ± 0.65
Secondary school 54 1.71 ± 0.77
High school 200 1.57 ± 0.61
College or University 211 1.52 ± 0.60
Postgraduate 26 1.64 ± 0.75
Income level
***
Very high 19 1.38 ± 0.67 F: 7.198 <0.001
High 268 1.48 ± 0.56
Moderate 331 1.71 ± 0.68
Low 17 1.66 ± 0.72
Occu pation
Worker (blue col l ars) 47 1.48 ± 0.49 F: 1.813 0.094
Civil servant 47 1.41 ± 0.54
Retired 109 1.70 ± 0.68
Self-emp loyed 73 1.61 ± 0.62
Housewife 199 1.64 ± 0.66
Student 17 1.75 ± 0.79
Others 143 1.56 ± 0.65

Notes. F: one-way ANOVA; t: t test.

*
p < 0.05**p < 0.01***p < 0.001

relation to those observed for patients admitted from the patient

3.4 | Comparison of PSNCQQ scores according to
admission department and through other means (1.54 SD 0.62, 1.58
patients’ medical histories
SD 0.66, respectively; p < 0.001 ).

The mean PSNCQQ score of the patients hospitalized in the inter-
nal medicine unit was higher (1.95 SD 0.75) than those of the pa-
tients in the surgery and the obstetrics and g ynaecolog y units (1.51
SD 0.57, 1.46 SD 0.55, resp ectively), and the differences were sig-
nificant (p < 0.001). Score for patients admitted to the service from
the emergency department was significantly higher (1.90 SD 0.69)
The mean score of the patients who had been hospitalized twice
in the preceding 2 years was higher (1.70 SD 0.71) than those of
the patients who had only been hospitalized once and more than
five times (1.55 SD 0.61, 1.35 SD 0.57, respectively), and the differ-
ences were significant (p < 0.001). The mean scores of the patients
who perceived their health status very poor, poor, fair and good
KARACAAn d d UR nA 541

TABLE 4 Comparison of Patient

Statistical eval u ation
Satisfaction with Nursing Care Quality
Questionnaire scores according to N M±SD t/F p
patients’ medical histories (N = 635)
Unit in which the patient is hospitalized
***
Surgery 212 1.51 ± 0.57 F: 36.35 <0.001
Obstetrics and gynaecology 261 1.46 ± 0.55
Internal medicine 162 1.95 ± 0.75
Manner of admission to ward
***
From emergency department 70 1.90 ± 0.70 F: 4.80 0.001
Directly from patient 391 1.54 ± 0.62
admissions department

After daily operations and 52 1.67 ± 0.68

tests
Transfer from another 28 1.69 ± 0.56
heal thcare facility

Other 94 1.58 ± 0.66

Status of hospitalization in the last 2 years
***
Once 423 1.55 ± 0.61 F: 3.90 0.004
Tw i ce 143 1.70 ± 0.71
Three times 42 1.81 ± 0.66
Four times 9 1.94 ± 0.67
Five times or more 18 1.35 ± 0.57
Percei ved heal th
***
Excellent 68 1.31 ± 0.55 F: 4.91 <0.001
Good 314 1.60 ± 0.64
Fair 160 1.65 ± 0.66
Poor 63 1.67 ± 0.64
Very poor 22 1.99 ± 0.69
Unsure 8 1.59 ± 0.71

Note. F: one-way ANOVA; t: t test.

*
p < 0.05**p < 0.01***p < 0.001

were higher (1.99 SD 0.69, 1.67 SD 0.64, 1.65 SD 0.66 and 1.60 SD decisions and reassurance (Goh et al., 2016; Sh in de & K ap u r k ar,
0.64, respectively) than in excellent health (1.31 SD 0.55), and the 2014). Effecti ve and continuous i nter acti on and comm u nicati on
differences were significant (p < 0.001). Also, the results showed are critical determinants in patients' satisfaction, hospital stay
that the duration of hospitalization was significantly associated and recover y (Koç, Sağlam, & Şenol, 2011; Mohanan et al., 2010;
with PSNCQQ scores; however, the correlation was weak (rp: 0.195, N eg ar and eh , Bahabad i , & Mam ag hani , 2014; V i l l ar r uz-Su l i t et al .,
p < 0.01; Table 4). 2009). Health p rofessi onal s’ com m unication skills play a p i votal
role in ensuring that patients feel valued and cared for. The alloca-
tion of sufficient time for talking and listening to patients and pro-
4 | DISCUSSION viding information is a prerequisite for patient satisfaction, as it
ensures that patients are less stressed and more engaged and well
The results of this study revealed similarities and differences with the adjusted (Koç et al., 2011). There is evidence that the health pro-
existing national and international literatures. This issue has been fessi onal s are perceived as com m unicating well when the p ati ent
discussed as follow. feels he/she shows individualized interest, understanding and
reassurance (Sitzia & Wood, 1997). A study (Abdel Maqsood et
al., 2012) in dic ated that p ati ents were more satisfied with hav i ng
4.1 | Discussion of finding about the
resp ectfu l communication whereas they were l ess satisfied wi th
PSNCQQ scores
the professional information provided by the nurses about the ir
Measures of patient satisfaction can assess communication in the disease, health status, investigations and prognosis of their condi-
consu l tati on such as inform ation transfer, p ati ent involvem ent i n tion. In a meta-analysis conducted by Özsoy et al. (2007), patients
KARACAAn d d UR nA
expected favour, attention, understanding, kindness and helpful-
ness from individuals providing care services. In our study, the
highest level of satisfaction, represented by PSNCQQ scores, was
reported for the “ Concern and Caring by Nurses” item. The results
indicate that the nurses’ communication style is to treat patients
resp ectfu l l y and be friendly towards them. However, the nurses
were less interested in explanations about their inter ventions and
communication with patients that did not meet their expectations.
Information provision and education are important factors affect-
ing patient satisfaction (Abdel Maqsood et al., 2012; Koç et al., 2011;
Villarruz-Sulit et al., 2009). Nurses and other healthcare profession-
als play a key role in providing support and information. Nurses care
for the patients on a 24-hr basis and should be empowered to pro-
vide requisite information and instructions to the patients (Alhusban
& Abualrub, 2009; Shinde & Kapurkar, 2014). Patient education has
been linked with positive clinical outcomes such as improved adher-
ence to a therap eutic regim e, reduced anxiety and enhanced ability
to cope with symptoms (Sitzia & Wood, 1997). It is known that re-
ceipt of adequate inform ation affects patients’ confidence and satis-
faction and this is the most important factor in encouraging patients
to participate in their own health care. In addition, providing patients
and their families with information about patients’ conditions is im-
portant in helping them overcome fear of the unknown (Dzom eku et
al., 2013; Koç et al., 2011; Milutinovic et al., 2012). Several studies
have reported inadequacies in information provision. For exam ple,
Dzomeku et al. (2013) found that the type and amount of information
provided by nurses about patients’ conditions constituted one of the
main causes of dissatisfaction. In a meta-analysis conducted by Özsoy
et al. (2007), the patients’ most im portant expectation concerning
care quality was that they should be informed about medication and
treatment. Patients reported that inform ation played an im por tant
role in their satisfaction and they em phasized that inform ation pro-
vided by nurses should be clear and concise. Therefore, it is crucial for
nurses to realize that information provision and education are nursing
responsibilities and that they should collaborate with other health-
care staff to provide complete and relevant information to patients.
Abdel Maqsood et al. (2012) indicated that the patients had low levels
of satisfaction with information and instructions given by nurses and
nurses had the perception that “ inform ation giving” was the role of
the physicians and the nurses may be f earful to provide information
because of the power hierarchy between the nurses and the physi-
cians. In this study, the lowest level of satisfaction, represented by
PSNCQQ scores, was reported for the “ Inform ation You Were Given”
explanations were about tests, treatments and what to expect” item.
This result indicates that the explanations and information provided
by nurses at the hospital were unsatisfactory in the nursing care.
4 .2 | Discussion of finding about to the PSNCQQ
scores according to patients’ socio-demographic
characteristics
Different studies indicated that older patients are generally more satis-
fied (Dzom eku et al., 2013; Fröjd et al., 2011; Milutinovic et al., 2012).
542
Sitzia and Wood (1997) stated in their review study that older people
tend to be more satisfied with health care than younger people are.
Similarly, according to Shinde and Kapurkar (2014) older respondents
were more satisfied, probably because they were more social and ac-
cepting than younger or they had more respect and care for providers.
On the contrary, we found that patients aged 56 years or older were
less satisfied than other age groups. This can be related to the fact that
the nurses did not pay more attention to elderly patients. Another pos-
sible reasons can be that levels of s atisfaction could differ according to
cultural values or the patients did not held positive attitudes towards
events, based on age-related increases in tolerance and maturity levels.
Similarly, in our study, Sitzia and Wood (1997) found that patient
gender d id not affect satisfaction values and a conclusion reached
also in the reports that significantly more men than women. In other
studies (Alsaqri, 2016; Arslan & Kelleci, 2011), no relationships were
found between gender and pati ent satisfaction levels. However,
while some of these studies (Akın & Erdoğan, 2007; Alhusban &
Abualrub, 2009) reported that women’s levels of satisfaction with
care were higher relative to those observed in men, others (Koç et
al., 2011; Milutinovic et al., 2012; Shinde & K ap u rk ar, 2014) showed
higher satisfaction levels in men relative to those observed in
women. In addition, in a study conducted by Dzomeku et al. (2013),
38% and 30% of hospitalized men and women, respectively, were
com pletely satisfied with their nursing care. While the reason for
these differences can involve cultural characteristics, they can also
occur because, relative to men, women pay more attention to hy-
giene andcare andare more anxious.
In this study, college or university graduates were more satisfied
relative to those who were literate patients. However, in some other
studies (Dzomeku et al., 2013; Geçkil et al., 2008; Milutinovic et al.,
2012; Özsoy et al., 2007), literate individuals and prim ar y school
graduates reported greater satisfaction with nursing services rela-
tive to that reported by college or university graduates. In addition,
Sitzia and Wood (1997) indicated that greater satisfaction was asso-
ciated with lower levels of education. Patients with lower levels of
education being most satisfied, similarly, showed that higher educa-
tional attainment was strongly associated with dissatisfaction. Some
studies (Akın & Erdoğan, 2007; Shinde & Kapurkar, 2014) revealed
that the level of education was not associated with patient satisfac-
tion. These study findings indicated that patients expect more from
nursing and care as their education levels increase. This can occur
because patients with high educational levels possess more informa-
tion about treatment alternatives and expect higher care standards
and therefore are more critical in this regard.
Patients with high incomes tend to anticipate an improvement
in their symptoms and expect to receive care from highly qualified
staff and they become dissatisfied if they receive care that does not
meet their expectations. Patients with low incomes had low health,
get lower health care, had less continuous relation with doctors and
have difficulties in getting appointments (Shinde & Kapurkar, 2014).
Some studies (Akhtari-Zavare et al., 2010; Arslan & Kelleci, 2011;
Özsoy et al., 2007) reported that satisfaction with nursing care did
not differ significantly according to incom e. In our study, patients
KARACAAn d d UR nA
with high incomes were more satisfied relative to those with mod-
erate incomes. We can say that these patients received care in the
direction of their expectations.
4.3 | Discussion of finding about to the PSNCQQ
al.
scores according to patients’ medical histories
charge report greater satisfaction than those with poor health sta-
Patients who were hospitalized in surgery and obstetrics and gy-
naecology units were more satisfied relative to those hospitalized
in the internal medicine unit. Shinde and Kapurkar (2014) found that
the gynaecological ward had a significantly higher percentage of
patients’ satisfaction with nursing care than the surgical wards. In
a study conducted by Alhusban and Abualrub (2009), the patients
hospitalized in an obstetrics and gynaecology unit reported higher
satisfaction levels relative to those hospitalized in internal m edicine
and surgical units, while in a study conducted by Geçkil et al. (2008),
patients hospitalized in surgical units reported higher satisfaction
levels relative to those hospitalized in obstetrics and internal medi-
cine units. In the other studies ( A kın & Erdoğ an, 2007; Koç et al., 2011;
Tang et al., 2013), satisfaction scores for p atients treated in in-
ternal m edicine units were higher relative to those treated in surgery
units. The difference in dissatisfaction between the types of units
occurred because of problems experienced during surgical pr oce-
dures in conjunction with medical diagnoses and socio-demographic
characteristics. A ll of these differences can be the levels of physical and
psychological dependency on the hospital.
The results of the present study revealed that the patients who
hospitalized once or at least five times in the preceding 2 years
were more satisfied relative to those hospitalized twice in the pre-
ceding 2 years. Alsaqri (2016) showed that there was a statistically
significant difference between previous admissions and patient
satisfaction levels. The same study demonstrated patients with
a history of admission to hospital during the last 2 years found
nurses more caring. It seems that more lengths of stay in hospital
increase patients’ opportunities for receiving more nurses’ care
and obser ving their caring behaviours. Similarly, in these studies
(Koç et al., 2011; M ilutinovic et al., 2012) sati sfacti on levels re- The results revealed that nurses should inform patients about each
ported by patients who had been hospitalized previously w ere
higher relative to those who had not. In contrast, in a study con-
ducted by Arslan and Kelleci (2011) satisfaction levels reported by
patients with previous hospital experience were lower relative to
those without previous hospital experience. The result of another
study (Akın & Erdoğan, 2007) found no statistical relationship
between satisfaction with nursing care and the numbers of hos-
pitalization. According to these results, we can say that patients’
exp ectati ons can v ar y according to prev ious exp eri ence in si m i-
lar situations and as the number of admissions increase, they can
compare their care with that received previously. Also, the positiv-
ity or negativity of patients’ previous experience can be reflected
in their approach to current care.
A study (Alsaqri, 2016) indicated that p eople who perceived
them selves as being healthy were more likely to be satisfied with
access to care. A ccording to A lsaqri (2016), p atients who p erceived
543
them selves to be in excellent or good health are more likely to be
satisfied with their health care. Also, it is indicated in the same study
that, a person’s health prior to arrival at hospital, whether through
accident, a chronic condition or a voluntary procedure may affect
the patients’ expectations about the care. In addition, Laschinger et
(2005) reported that patients with good heal th status postdis-
tus. Similarly, in our study, patients with very poor, poor, fair or good
health were less satisfied relative to those of patients with excellent
health. This m ay be due to the fact that healthier people do not need
as much medical care and they interact with healthcare providers
less frequently. They have less opportunity to exp erience problem s
with access to health care and therefore may express more satisfac-
tion with access.
4.4 | S tu d y l i m i tati ons
The sample was restricted to patients from the general surgery, ob-
stetrics and internal m edical units. In addition, the study was con-
ducted in a single private hospital in Turkey. Therefore, the results
cannot be generalized to all hospitals. Future studies should include
more than one hospital in both the private and public sectors and
the nursing care provided in private and public hospitals should be
compared.
Test-retest reliability analysis should have been performed to
strengthen the results of the study. Therefore, patients should be
surveyed for a second time in 2 weeks of discharge and the results
should be tracked and addressed in future studies. Although meth-
odological problems, such as poor return rates and an inability to
collect tracking data for all p ar ticip ant s occurred in the study, the
results could be considered useful because of the stability criterion for
p atient satisfaction surveys.
5 | CONCLUSION
application and procedure and provide necessary explanations about
illness, diagnosis and treatment to ensure patient satisfaction and
the provision of high-quality nursing care. The results also showed
that nurses should provide care in a framework of respect, favour
and courtesy towards patients by emphasizing the importance of
communication. Besides these, the patients were highly satisfied
with overall quality of hospital care, nursing care and reported that
they would recommend this hospital to their families and friends.
Nurse managers could contribute to the quality service provi-
sion by evaluating the patient satisfaction with nursing care for the
develop m ent and im provem ent of nur sing care based on patients’
exp ectations. Data obtained from this evaluation should be consid-
ered in determining training requirements for nurses and in-service
training programs should be organized to develop nurses’ knowl-
edge and skills in care planning. The PSNCQQ is considered useful
for nurse ad m inis tr ato rs in im provin g n u rsin g care. The scale could
KARACAAn d d UR nA
allow managers to determine the attitudes of individuals with whom
they work and those whom they manage and exert some degree of
control over employees’ behaviour.
ACKNOWLEDGEMENTS
The authors thank the patients who participated into the study.
CONFLICT OF INTEREST
The authors declare that there was no conflict of interests.
AUTHOR CONTRIBUTIONS
AK, ZD: Studydesign. AK, ZD: Data collection andanalysis. AK, ZD:
Manuscript preparation.
ORCID
Anita Karaca https://orcid.org/0000-0001-6552-4399
REFERENCES
Abdel Maqsood, A. S., Oweis, A. I., & Hansa, F. S. (2012). Differences
between patients’ expectations and satisfaction with nursing care in
a private hospital in Jordan. International Journal of Nursing Practice,
18, 140-146. htt ps://doi.org/10.1111/j.1440-172X. 2012.02008.x
Akhtari-Zavare, M., Abdullah, M. H., Syed Hassan, S. T., Binti Said, S.,
& Kamali, M. (2010). Patient satisfaction: Evaluating nursing care for
patients hospitalized with cancer in Tehran Teaching Hospitals, Iran.
Global Journal of Health Science, 2 (1), 117-126. https://doi.
org/10.5539/gjhs.v2n1p117
Akın, S., & Erdoğan, S. (2007). The Turkish version of the newcastle
satisfaction with nursing care scale used on medical and surgi-
cal patients. Journal of Clinical Nursing, 16, 646-653. https://doi.
org/10.1111/j.1365-2702.2006.01583.x
Alhusban, M. A., & Abualrub, R. F. (2009). Pati ent satisfaction w ith
nursing care in Jordan. Journal of Nursing Management, 17, 749-758.
https://doi.org/10.1111/j.1365-2834.2008.00927.x
Alsaqri, S. (2016). Pati ent satisfaction with quality of nursing care at
governmental hospitals, Ha’il City, Saudi Arabia. Journal of Biology,
Agricultureand Healthcare, 6(10), 128-142.
Arslan, Ç., & Kelleci, M. (2011). Bir üniversite hastanesinde yatan hasta-
ların hemşirelik bakımından memnuniyet düzeyleri ve ilişkili bazı fak-
törl er [Satisfaction levels of hospitalized pati ents in care of nursing in a
university hospital and some related factors]. Anadaolu Hemşirelik Ve
Sağlık Bilimleri D ergisi, 14, 1-8.
Buchanan, J., Dawkins, P., & Lindo, J. L. M. (2015). Satisfaction with nurs-
ing care in the emergency department of an urban hospital in the
developing world: A pilot study. International Emergency Nursing, 23, 218-
224. org/10.1016/j.i enj.2015.01.001
Dzomeku, V. M., Atinga, B. E., Tulukuu, P., & Mantey, R. E. (2013).
In-patient satisfaction wıth nursing care: A case study at Kwame
Nkrumah University of Sc ience and Technology hospital.
International Journal of Research in Medical and Health Sciences, 2(1),
19-24.
Freitas, J. S., Silva, A . E. B. C., Minamisava, R., Bezerra, A . L . Q.,
& Sousa, M. R. G. (2014). Quality of nursing care and satis-
faction of patients attended at a teaching hospital. Revista
544
L ati no-A meri cana De Enfermagem, 22(3), 454-460. https://d oi .
org/10.1590/0104-1169.3241.2437
Fröjd, C., Swenne, C. L., Rubertsson, C., Gunningberg, L., & Wadensten,
B. (2011). Pati ent inform ation and p ar ticip atio n s till in need of
improvement: Evaluation of patients’ perceptions of quality of
care. Journal of Nursing Management, 19, 226-236. https://doi.
org/10.1111/j.1365-2834.2010.01197.x
Gadalean, I., Cheptea, M., & Constantin, I. (2011). Evaluation of patient
satisfaction. Applied Medical Informatics, 29(4), 41-47.
Geçkil, E., Dündar, Ö., & Şahin, T. (2008). Adıyaman il merkezindeki
hastaların hemşirelik bakımından memnuniyet düzeylerinin değer-
lendirilmesi [Evaluation of patients’ satisfaction levels from nursing
care at the centre of the city Adiyaman]. Sağlık Bilimleri Fakültesi
Hemşirelik Dergisi, 15(2), 41-51.
Goh, M. I., A ng, E. N. K., Chan, Y. H., He, H. G., & Vehvilainen Julkunen,
K. (2016). A descriptive quantitative study on multi-ethnic patient
satisfaction with nursing care measured by the revised humane
caring scale. Applied Nursing Researc, 31, 126-131. https://doi .
org/10.1016/j.apnr.2016.02.002
Koç, Z., Sağlam, Z., & Şenol, M. (2011). Patient satisfaction with the nurs-
ing care in hospital. Türkiye Klinikleri Journal ofMedical Sciences, 31(3),
629-640. https://doi.org/10.5336/medsci.2009-16413
Ksykiewicz-Dorota, A., Sierpińska, L., Gorczyca, R., & Rogala-Pawelczyk,
G. (2011). Polish version of patient satisfaction with nursing care
quality questionnaire (PSNCQQ). Progress in Health Sciences, 1(1),
51-56.
Laschinger, H. S., Hall, L. M., Pedersen, C., & Almost, J. (2005). A psycho-
metric analysis of the Patient Satisfaction with Nursing Care Quality
Questionnaire. Journal of Nursing Care Quality, 20(3), 220-230.
https://doi.org/10.1097/00001786-200507000-00006
Merkouris, A., Andreadou, A., Athini, E., Hatzimbalası, M., Rovithis, M., &
Papastavrou, E. (2013). Assessment of patient satisfaction in public
hospitals in Cyprus: A descriptive study. Health ScienceJournal, 7(1),
28-40.
Milutinovic, D., Simin, D., Brkic, N., & Brkic, S. (2012). The pa-
tient satisfaction with nursing care quality: The psycho-
metric study of the Serbian version of PSNCQ question-
naire. Scandinavian Journal of Caring Sciences, 26, 598-606.
org/10.1111/j.1471-6712.2012.00969.x
Mohanan, K., Kaur, S., Das, K., & Bhalla, A. (2010). Patient satisfaction
regarding nursing care at emergency outpatient department in a ter-
ti ar y care hospital. Jour nal of M ental H ealth & H uman Behavi or, 15(1),
54-58.
Negarandeh, R., Bahabadi, A. H., & Mamaghani, J. A. (2014). Impact of
regular nursing rounds on patient satisfaction with nursing care.
Asian Nursing Research, 8, 282-285. https://doi.org/10.1016/j.
anr.2014.10.005
Özer, A ., & Çakıl, E. (2007). Sağlık hizmetlerinde hasta memnuniyetini et-
kileyen faktörler [The factors affecting patient satisfaction in heaath
services]. Tıp Araştırmaları Dergisi, 5(3), 140-143.
Özsoy, S. A., Özgür, G., & Durmaz Akyol, A. (2007). Patient expec-
tation and satisfaction with nursing care in Turkey: A literature
review. International Nursing Review, 54, 249-255. https://doi.
org/10.1111/j.1466-7657.2006.00534.x
Reck, D. L. (2013). Can and should nurses be aware of patients’ expec-
tations for their nursing care? Nursing Administration Quarterly, 37(2),
109-115. https://doi.org/10.1097/N A Q. 0b013e3182869e03
Senarath, U., & Gunawardena, N. S. (2011). Development of an instru-
ment to measure patient perception of the quality of nursing care and
related hospital services at the national hospital of Sri Lanka. Asian
Nursing Research, 5(2), 71-80. https://doi.org/10.1016/
S1976-1317(11)60015-7
Shinde, M., & Kapurkar, K. (2014). Patient’s satisfaction with nursing
care provided in selected areas of tertiary care hospital. International
Jour nal of Science and Research, 3 (2 ) , 150-16
KARACAAn d d UR nA 545
 KARACAAn d d UR nA 546

Şişe, Ş. (2013). H astal arı n hemşi rel i k hizmetl eri nden m em nuniy eti
[Satisfaction of patients with nursing care]. Kocatepe Tıp Dergisi, 14,
69-75.
Sitzia, J., & Wood, N. (1997). Patient satisfaction: A review of issues and concepts. Social Science and Medicine, 45(12), 1829-1843.
https://doi.org/10.1016/S0277-9536(97)00128-7
Tang, W. M., Soong, C. Y., & L im , W. C. (2013). Pati ent satisfaction w ith nursing care: A descriptive study using inter action m odel of
client health behavior. International Journal of Nursing Science, 3(2), 51-56. https://doi.org/10.5923/j.nursing.20130302.04
Villarruz-Sulit, M. V. C., Dans, A. L., & Javelosa, M. A. U. (2009). Measuring satisfaction with nursing care of patients admitted in the medical
wards of the Philippine general hospital. Acta Medica Philippina, 43(4), 52-56.
You, L., Aiken, L. H., Sloane, D. M., Liu, K., He, G., & Hu, Y. (2013). Hospital nursing, care quality and patient satisfaction: Cross-sectional
surveys of nurses and patients in hospitals in China and Europe. International Journal of Nursing Studies, 50, 154-161.
https://doi.org/10.1016/j. ijnurstu.2012.05.003
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

H ow to ci te thi s arti cl e: K araca A , Durna Z. Patient

satisfaction with the quality of nursing care. Nursing Open.
2019;6:535-545. https://doi.org /10.1002/nop2.237
 KARACAAn d d UR nA 547

International Journal of Health Care Quality Assurance

The nursing quality indicator framework tool
Dawn Connolly, Fiona Wright,
Article information:
To cite this document:
Dawn Connolly, Fiona Wright, (2017) "The nursing quality indicator framework tool", International
Journal of Health Care Quality Assurance, Vol. 30 Issue: 7, pp.603-616, https://doi.org/10.1108/
IJHCQA-08-2016-0113
Permanent link to this document:
https://doi.org/10.1108/IJHCQA-08-2016-0113
Downloaded on: 18 September 2017, At: 07:34 (PT)
References: this document contains references to 45 other documents.
To copy this document: permissions@emeraldinsight.com
The fulltext of this document has been downloaded 180 times since 2017*
Users who downloaded this article also downloaded:
(2017),"Applying adult development theories to improvement science", International Journal
of Health Care Quality Assurance, Vol. 30 Iss 7 pp. 617-627 <a href="https://doi.org/10.1108/
IJHCQA-09-2016-0124">https://doi.org/10.1108/IJHCQA-09-2016-0124</a>
(2017),"Brand trust and image: effects on customer satisfaction", International Journal of
Health Care Quality Assurance, Vol. 30 Iss 7 pp. 580-590 <a href="https://doi.org/10.1108/
IJHCQA-04-2016-0054">https://doi.org/10.1108/IJHCQA-04-2016-0054</a>

Access to this document was granted through an Emerald subscription provided by

Token:Eprints:ZWUDM9D98Q2328URVWHI:
For Authors
If you would like to write for this, or any other Emerald publication, then please use our Emerald
for Authors service information about how to choose which publication to write for and submission
guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
Emerald is a global publisher linking research and practice to the benefit of society. The company
manages a portfolio of more than 290 journals and over 2,350 books and book series volumes, as
well as providing an extensive range of online products and additional customer resources and
services.
Emerald is both COUNTER 4 and TRANSFER compliant. The organization is a partner of the
Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for
digital archive preservation.

*Related content and download information correct at time of download.

 KARACAAn d d UR nA 548

The current issue and full text archive of this journal is available on Emerald Insight at:
www.emeraldinsight.com /0952-6862.htm

The nursing qualityindicator framework tool

NQI

framework tool
Dawn Connolly and Fiona Wright
Southern Health and Social Care Trust, Northern Ireland
603
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

Abstract Received 17 August 2016

Purpose - The purpose of this paper is to develop a nursing quality indicator (N Q I) framework and provide a
comprehensive reporting mechanism for nursing care. Revised 22 March
2017
Design/methodology/approach - Mixed method, including patient records audit, patient experience Accepted 5 May 2017
questionnaire, nurse self-report questionnaire and collecting ward-level information. The sample was 53
patients and 22 nurses.
Findings - Outputs from the NQ I framework domains offer a more comprehensive understanding of nursing
quality compared to when domains are analysed separately. The NQI framework also provides a more
inclusive mechanism for assuring nursing care.
Research limitations/implications - Sample size was limited to 53 English-speaking patients who
consented to participating in the study.
Originality/value - One design strength was the ability to describe individual patient care across the four
domains and subsequently show relationships between nursing knowledge, nursing interventions and patient
outcomes/experiences. Additionally, corroborated information from three sources (documentation review,
patient and nurse responses) strengthened the conclusion that the NQI framework could provide more
comprehensive assurances on nursing quality and identify care improvements.
Keywords Patient experience, Nurse’ s knowledge, Nursing quality indicators, Structure-process-outcome
Paper type Research paper

Introduction
In performance-managed health services across the world, there is an emphasis on
“Ward-to-Board” accountability for nursing quality. Although healthcare systems may differ,
all have a common goal: to improve service quality. In the 1990s, healthcare reform in the USA
prompted nursing quality indicator (N Q I) development. Databases such as the Californian
Nursing Outcomes Coalition and the National Database of Nursing Quality IndicatorsTM
®
(NDNQI ) incorporated executive and clinical information necessary for reporting on quality
assurance ( Montalvo, 2007; Aydin et al., 2 008). Over the past decade in the U K, measuring
compliance by applying well-defined indicators has supported professional transparency,
accountability and quality improvement (NHS Quality Improvement Scotland, 2005;
Welsh Assembly Government, 2010; Northern Ireland Practice Education Council, 2011;
Department of Health, Social Services and Public Safety, 2011). Measuring care based on
patient experience is a relatively new consideration and is now included in many nursing care
indicators (Maben et al., 2012; McCance et al., 2012). Different quality measures allow
managers to articulate the nursing pr ofession’ s contribution to quality care in tangible terms
and assure hospital boards that the profession is providing safe, effective, person-centred care.
NQIs also focus development activities in areas that are aligned to policyand organisational
imperatives, which aim to proactively improve service qua lity. Including NQIs within
performance management systems provides a robust framework that can support delivery
and assurance on clinical and social care governance ( M cCa nce et al., 2012). However,
in practice, articulating nursing value through quality indicators is both a challenge and an
opportunity for the profession to accurately apply indicators, i.e., obtaining empirical evidence
is far from straightforward (Burston et al., 2013). Defining potential indicators, demonstrating
associations between indicators and nursing care, collecting and analysing data, and sharing International Journal of Health
the outcomes is complex (Doran et al., 2006; Needleman et al., 2009; Burston et al., 2013; Care Quality Assurance
Vol. 30 No. 7, 2017
Heslop and Lu, 2014). pp. 603-616
© Emerald Publishing Limited

0952-6862
DOI 10.1108/IJHCQA-08-2016-0113
 IJHCQA Qualityhealthcare dimensions that inform the NQI framework
30,7 The American Nurses Association developed the ND N Q I ® , grounding it on the
Donabedian framework (Gallagher and Rowell, 2003; Montalvo, 2007). Donabedian’s
(1988) conceptual model provides a framework for examining healthcare quality through
structure, process and outcome. He believed that care quality is not only reflected in each
individual category, but also in the relationship between the m. Several widelyrecognised
nursing structure, process and outcome indicators are cited in the literature including:
604 nurse-to-patient ratio, sickness and absences, registered nurse education level and
experience, hospital acquired infection, pressure ulcers, falls and medication
administration (Griffiths et al., 2008; Maben et al., 2012). The extent to which these
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

indicators are sensitive to nursing quality variation is unclear (Savitz et al., 2005;
Heslop and Lu, 2014) and inconsistent associations have been identified between
structural measures and patient outcomes (Blegen and Vaughn, 1998; Aiken et al., 2002;
Needleman et al., 2002; McGillis Hall et al., 2004). Nonetheless, these pointers are frequently
acknowledged as plausible NQIs within healthcare organisations (Maben et al., 2012).
Griffiths et al. (2008) consider that patient outcomes are best reflected in their experience
of compassionate nursing care. Person-centred care that is respectful, compassionate and
responsive to individuals is recognised as a key quality indicator and an essential
component to strive for when improving healthcare systems (US Institute of Medicine,
2001; De Silva, 2014). Failure to listen to patient and relative experiences has been
implicated in investigations as a key factor in failing hospitals ( Francis, 2013). In a
systematic review, Doyle et al . (2013) suggest that patient experience data, robustly
collected and analysed, increases the likelihood that patient safety and clinical
effectiveness improve. This supports the view that safety, effectiveness and patient
experience indicators should be considered together and not in isolation.
Donabedian’ s model does not include antecedent characteristics that are important
precursors to evaluating service quality (Coyle and Battles, 1999). The person-centred
framework developed by McCormack and McCance (2010) consider nurse attributes,
including professional competence, as a pre-requisite to person-centred outcomes. Nurses
are expected to know their patients ’ nursing care needs and will apply this in professional
decision making to deliver safe, effective and person-centred care. The public also expect
nurses to demonstrate professional competence, sound clinical judgement and decision
making (American Nurses Association, 2015; Nursing and Midwifery Council, 2015).
Therefore, it is reasonable that the nurse ’s knowledge of patient’ s nursing care needs should
be considered as a NQI. The contention is that if a nurse assesses and delivers nursing care
appropriate to patient needs, then it is likely/expected that the patient will experience
positive outcomes from that care. Overall the nursing literature indicates that: first, no single
measure can give a complete picture; second, patient experience is an outcome; third, safety,
effectiveness and patient experience should be considered together; and finally, nurse
attributes, including professional competence, are important prerequisites to safe, effective
person-centred care.

NQI framework domains

The NQI framework domains are drawn from the literature and based on the principle
that optimal high-quality nursing can only be achieved if all elements (safe, effective,
person-centred care) are present equally and simultaneously. This supports Donabedian’ s
view that service quality is related to structure, process and outcome elements individually
and to relationships between them. The NQI framework strengthens patient-level data
analysis by linking nursing care elements related to structure, process and outcome, patient
experience and nurses’ knowledge. Patient-level data can be aggregated to ward and board
 level to assure nursing care. The NQI framework includes four discrete but complementary NQI
domains (Table I): framework tool
(1) Safe and effective process indicators (SEPIs) were based on the UK nationally
approved care bundles, i.e., national early warning score (NEW S), FallSafe, SKIN
(surface, keep moving, incontinence, nutrition), Malnutrition Universal Screening
Tool ( MUST bundle) (Royal College of Physicians (R C P), 2012a, b; NHS Wales, 2010;
BAPEN, 2011). 605
(2) Safe and effective outcome indicators (SE O I) were linked to process indicators and
based on the premise that there is a relationship between processes and outcomes.
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

If the care bundle processes are consistently and reliably applied, then this should
result in better patient outcomes, e.g., reliably applying the SKIN bundle processes
should prevent a patient from developing a pressure ulcer.
(3) Patient experience indicators were developed from primary research carried out by
McCance et al. (2012), where eight key performance indicators, focusing on unique
nursing/midwifery contributions to the patient experience, were identified using a
consensus approach.
(4) Nurse’s knowledge of patient’s nursing care needs: the nurse responsible for
the patient’ s care should be able to articulate the nursing care required to meet the
patient’ s needs. The nurse caring for the patient will apply his/her knowledge to
deliver safe, effective and person-centred care to meet those needs.
The NQI framework combines process, outcome, patient experience and nurses’ knowledge
indicators as applied to individual patients.

Methodology
Aim and objectives
We aimed to examine the NQI framework as a mechanism for reporting assurances that
nursing care was safe, effectiveand person-centred. Our objectives were to:
(1) undertake a nursing records analysis to determine compliance with agreed
evidence-based care bundles (i.e . pressure ulcers, falls, nutrition, omitted medicines
and identifying the deteriorating patient);
(2) determine nursing impact using SEOI;
(3) gather information on the patient ’ s nursing experience during their stay, collected
through patient stories and analysing patient experience indicators;

NQI framework: nurses’ contribution to safe, effective, person-centred care

Domain 3 Domain 4
Domain 1 Domain 2 Patient Nurse’s knowledge of the
Safe and effective Safe and effective experience patient’s nursing care
process indicators outcome indicators indicators needs indicators

Patient-level Review of patient Review of patient records Exploration of Nurse’s knowledge of

data records to assess to determine patient patient’s patient’ s nursing care
compliance with safety outcomes in perception of needs. Responses mapped
evidence-based care relation to the selected their experience against nursing care plan
bundles process indicators of nursing care and progress notes
Ward-level Patient safety outcome measures; feedback from nurses; complaints and incidents Table I.
information NQI framework
 IJHCQA (4) explore individual nurse ’ s knowledge of the patient ’ s nursing care needs and the
30,7 nurse’s experiences delivering care; and
(5) review incidents and complaints for the calendar month in which the study was
carried out.

606 Research setting

The study was conducted in one UK healthcare organisation using a mixed method,
case study approach. The focus for the study was nursing care delivered in acute and older
people’s wards. Three wards (medicine, surgical and older people) were selected. The data
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

collection period was one calendar month.

Sample size
Five patients were purposively selected each week from participating wards giving
20 patients per ward: a standard sample size for assessing compliance with care bundles
and in quality improvement projects that measure processes over time (Perla et al., 2013).
Nurses sampled were self-selected from those responsible for delivering nursing care to the
participating patients.

Inclusion/exclusion criteria
To be eligible, patients were required to be 16+ years, have capacity to give consent,
speak English, have been admitted to the participating ward for at least 24 hours and met
the criteria for at least four SEPIs. Acutely ill patients or those receiving end of life care
were excluded.

Participants
The ward sister/charge nurse identified patients meeting the inclusion criteria. After
explaining the study, patients were given time to decide whether they wished to
participate. Those who agreed completed a consent form. Nurses were recruited through a
self-selection process from those responsible for delivering nursing care to pa rticipating
patients. The researcher and ward sister/charge nurse agreed suitable dates and times for
data collection.

Data collection
Data collection included: first, auditing patient records in relation to SEPI and SEOI; second,
administering a patient experience questionnaire; and finally, running a self-report
questionnaire, which focused on the nurse’s knowledge of their patient’ s nursing care needs
and their care delivery experience. Process indicators were measured by reviewing patient
records to ascertain compliance with NEWS bundle for identifying deteriorating patient
(RCP, 2012a), FallSafe care bundle (nursing elements) (RCP, 2012b), SKIN care bundle
(Gibbons et al., 2006), MUST (BAPEN, 2011) and administering critical medicines (National
Patient Safety Agency, 2010). The related SEOI were linked to the process indicators and
measured by a patient records review. The outcome indicators included cardiac arrest or
unplanned admission to intensive care unit , fall or fall resulting in an injury,
hospital-acquired pressure ulcer (grade 2 and above), weight loss W5 per cent body
weight whilst in hospital and omitting a critical medicine dose. To facilitate analysis, data
were entered into a bespoke Excel ® macro-enabled spreadsheet. Patient experience
indicators were applied to Sensemaker ® software (a proprietary research method and tool
developed by Cognitive Edge, cognitive-edge.com) to produce a bespoke patient experience
questionnaire specific to this study (McCance et al., 2012). Patients were asked to summarise
 their nursing care experience in a story format. They were asked six questions (triads) NQI
with three pre-set responses. For each question patients were asked to place a dot within framework tool
the triangle that best reflected their experience. Where requested, the researcher
undertook to record the patient’ s experience and responses to the questions as directed by
the patient. Additionally, patients were asked to rank order care aspects most important
to them (pre-set responses were linked to the patient experience indicators) and to
describe their nursing care experience on a scale ranging from strongly positive 607
to strongly negative. Nurses caring for participating patients completed a short
questionnaire asking them to describe their patien ts’ nursing care needs during their
shift. Their responses were compared with entries made in the patient ’ s nursing care plan
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

and nursing progress notes. Nurses were asked how they felt the patient might describe
his/her nursing care experience. These responses were mapped against the patient ’ s
description using the same scale. Nurses were also given an opportunity to say what
would improve care experiences.

Study design
The study design involved data source triangulation (patients, nurses, records) and method
triangulation (questionnaires, documentation analysis). Triangulation facilitates cross-data
verification thus increasing credibility and validity ( Lincoln and Guba, 1985). Additionally, the
researcher adhered to a strict data collection process using regionally agreed guidance for
assessing consistency in care bundle application. If there were inconsistencies in application
or non-compliance with a bundle, then the researcher discussed and verified information with
the ward sister or specialist nurse.

Ethical considerations and research governance

We complied with the research governance framework for health and social care and good
clinical practice following approval from the Office of Research Ethics Committees Northern
Ireland and the H SC Trust Research Governance Committee. The risks to participants were
minimised by:
• providing them with information about the study and obtaining informed consent;
• ensuring confidentiality and anonymity, where possible; and
• having mechanisms in placeto deal with unforeseen issues that may arise in practice
during the survey, e.g., a Distress Protocol.

Limitations
Whilst this study sample was limited to English-speaking patients and able to give consent
to participate, any repeat studies should include a wider patient sample and ethnicities, and
vulnerable adults whose relatives/carers may wish to report care experience on the
patient’s behalf.

Findings
Demographics
The sample included 42 female and 11 male patients. Most patients were 70 years and
older (n ¼ 31) and had been nursed on the participating ward between four and seven
days (n ¼ 23). In total, 22 nurses (19 females and 3 males), returned the questionnaire
giving a 42 per cent response rate. Most responses were from nurses in the 18 to 30 a ge
band (n ¼ 10).
 IJHCQA SEPIs
30,7 Compliance with the SEPIs was measured using the associated care bundle for the
selected nursing care process. Resar et al. (2012) define a care bundle as evidence-based
interventions for a defined patient population and care setting, and proposes that,
when implemented together, result in better outcomes than when employed individually.
Applying the care process bundle aims to achieve 95 per cent compliance, hence
608 improving patient outcome (Resar et al., 2012). Compliance with bundles uses an
all-or-none measurement approach. If an individual element has not been recorded as
completed, then the whole bundle compliance will be scored as 0 per cent regardless
whether other elements have been documented as being completed (Resar et al., 2012).
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

Findings are presented in Table II.

We identified common departures from good record keeping across all process
indicators, including:
• associated care bundle charts not always initiated;
• charts not always reviewed in a timely fashion;
• monitoring frequency not always recorded;
• prescribed monitoring frequency was not adhered to or was not recorded as being
changed in line with observation;
• reason for non-compliance not always recorded;
• charts completion varied between wards and between individual nurses, e.g., a “No”
response could also have meant “Not applicable” (N A ); and
• the reason why a critical medicine was not administered was not always recorded.

SEOI
Patient outcomes were very positive/good (94 per cent) despite nursing records indicating
variable compliance with some care bundle elements. Three patients did not have good
outcomes; one who had been identified as risking malnutrition had a weight loss W 5 per cent,
one had a non-injurious fall and one developed a hospital-acquired pressure ulcer (grade 2)
during the hospital stay. In all three instances, the record audit showed non-compliance with the
associated care bundle.

Nursing quality Elements Records Total Individual elements Records with all care bundle
indicator (NQI) in bundle audited elements completed care elements completed

NEWS 6 53 318 91% ( n ¼ 288) 77% ( n ¼ 41)

FALLS 9 53 477 92% ( n ¼ 440) 47% ( n ¼ 25)
SKIN 14 16a 224 66% ( n ¼ 148) 0% (n ¼ 0)b
MUST 5 53 265 93% ( n ¼ 247) 79% ( n ¼ 42)
TOTAL 34 53 1,284 87% ( n ¼ 1,123)
Critical Records Doses Medicine doses administered Patients having all doses
medicinesc audited prescribed administered
53 1,027 98% ( n ¼ 1,011) 79% ( n ¼ 42)
Table II. Notes:aAs patient inclusion criteria; bcare bundles require an “all-or-none” measurement, i.e., if an individual
Safe and effective element has not been recorded as being completed, then the whole bundle compliance will be scored 0 per cent
process indicators regardless of whether all other elements have been documented as complete (Resar et al., 2012);cdoes not meet
(SEPIs) bundle definition
 Patient’ s nursing care experience NQI
Patient experience was gathered from patient stories, six completed questions (triads) and framework tool
ranking their nursing care experience according to what was most important to them:
(1) Patient stories
Patients reported that friendliness, kindness and timely actions were key to good
nursing care experiences. They valued the fundamental nursing care aspects that
focused on relational and functional care aspects. For example: 609
Very attentive nurses. They attend to my needs - taking me to the toilet, helping me dress, if
I need help with food - they help. Excellent help getting to bed. Prompt with tablets and
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

whatever we have to get (04A).

A major aspect was asking regularly whether I was in pain and offering pain relief.
The nursing staff were very caring and always listened (02C).
I was prepared for surgery very professionally and the procedure was very clearly
explained to me. Post-surgery nursing care has been excellent (20B).
Where patients rated their experience less positively, staffing levels, organisational
factors and difference between nurses, e.g., certain nurses were better than others at
giving care, were most often described.

(2) Completingthe six questions (triads) (see Figure 1).

(3) Ranking patient’ s nursing care experience.
When asked to rank which care aspects were most important to them, feeling safe whilst in
the nurse’s care, having confidence in the nurse’ s knowledge and skills and the nursing staff
having the same understanding, were the most frequently selected responses. This
validated the decision to include patient experience as a key domain in this NQI framework.
Most patients (89 per cent) reported that their nursing care experience was either strongly
positive or positive. Four patients (7 per cent) reported “neutral” feelings and only two
(4 per cent) were “not sure” about how they felt about their experiences.

Understanding the patient’s nursing care needs

Nurses’ responses to the questionnaire on their patients nursing care needs were compared with
entries made in individual patient’ s nursing care plans/progress notes. Findings show that the
patient’ s current nursing care needs were not always recorded in the nursing care plan/progress
notes, however, when asked to record on a blank page all nurses were clear about their patient’ s
current care needs at that time. Nurses were also asked how their patient might describe their
nursing care experience and these were mapped against the patients’ responses using the same
scale, i.e., ranging from strongly positive to strongly negative. Results showed that nurses
correctly predicted that their patients would say that they had a positive experience (91 per cent)
or neutral experience (9 per cent). Only two nurses and their patien ts’ responses did not
correspond. Nurses also had an opportunity to say what would improve care delivery. In total,
13 nurses (60 per cent) reported that they would like more time with their patients. The most
common suggestion to enhancing care delivery experience was less d ocumentation; however,
others felt that documenting patient care was an integral to nursing care and that freeing up time
to be with patients included the time needed to document care given. Another shared theme
related to staffing levels, specifically, better staffing levels to support patients who are restless/
confused and a falls risk, and more staff at weekends as the nursing care activity does not change:
More staff at weekends.There are no less nursing needs at the weekend so I don ’t understand why
there is less staff (05B).
 Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

Q1. How did you feel about the nurses’ Q2. How confident were you in the Q3. How safe did you feel while you were
understanding of the care you needed? nurse’s skills? being looked after by nurses?
I had confidence in the skills I felt safe all of
All of the nurses had a of all the nurses who were
good understanding of the time
looking after me
the care I needed

48 stories in
38 stories in this 43 stories in this cluster -
cluster - 72% of this cluster - 91% of this
this triad’s stories 81% of this
triad’s
are located in this triad’s
stories are
cluster stories are located in
located in this cluster
this cluster

All of the nurses had a different Some of the nurses had a good I had no confidence in the skills I had confidence in the It depended on who was
understanding of the care I understanding of the care I of any of the nurses who were skills of some of the nurses I always felt Vulnerable looking after me as to
needed needed looking after me who were looking after me and unsafe whether I felt safe or not

Q4. How did you feel about the time Q5. How would you describe the nurses’ Q6. How appropriate did you feel the
nurses spent with you? respect for your personal preferences and care you received was against the things
choices? which were important/relevant to you?
All nurses took account Care was always focused on
of my preferences and my needs and what was
I rarely saw the nurse choices important to me

46 stories in 41 stories in
this cluster - this cluster -
87% of this 77% of this
triad’s stories triad’s stories
are located in are located in
this cluster this cluster

44 stories in
this cluster-
83% of this
triad’s stories
are located in
this cluster

Care did not focus on The care I received

The nurse was with me I felt the staff gave me the None of the nurses respected It depended on who was
things that were important depended on the nurse
more than I wanted time when I needed it my preferences and choices looking after me
to me looking after me
 From nurses’ responses, findings indicate that at certain times increased workload and NQI
insufficient staffing levels affect staff well-being and nurses reported that at times they felt framework tool
frustrated that they could not give patients the nursing care they would like to:
At times feel very frustrated that I can ’ t give the care that I would like to give due to increased
pressures of workload such as restless patients decreased staff levels and the expectations of some
patients and relatives (05A).
611
Findings summary (all four domains)
Findings show that when quality determinants represented in the framework’ s four
domains were analysed collectively, they offer a more comprehensive understanding of
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

nursing care quality than when each was analysed and reported separately. Figure 2
represents the overall findings from one participating ward and provides a more
comprehensive and rounded nursing picture. The individual vertical columns set out the four
domains and the data analysis is colour coded red, amber green rating, with further
information embedded within the table cells to assist with interpretation.
Figure 2 shows that patient safety outcomes were good despite variable compliance with
recording some process elements, specifically in one care bundle. Patient experience was
positive and matched the nurse’ s predictions. When asked, the nurse ’ s knowledge of
patient’ s nursing needs for the shift was good even when care was not recorded in a formal
care plan. If the SEPI and SEOI, and the patients ’ experience and nurses’ knowledge are
considered separately, then the interpretation is different than when all are considered
together within the framework. In other words, seeing the whole picture provides greater
overall nursing quality understanding than when the domains are looked at individually.

Ward-level information
We carried out a retrospective incidents and complaints review for the calendar month to
check consistency with patient-level data and add rigour to data collection. There were
12 clinical incidents reported through the Datix Risk Management system; reports were
mapped to NQI outcome data.

Discussion
Our aim was to examine the NQI framework as a mechanism for assuring nursing care
quality. Assurance to boards is often reflected in performance levels. However,
compliance with processes does not necessarily mean that the patient experienced good
quality care. This study tested the NQI framework application by mapping patient ’ s care
experiences and outcomes against nurses’ knowledge and prescribed care interventions. We
found that when the NQI framework determinants were viewed together, they provided a more
comprehensive understanding than when considered separately.
The study followed the patient’ s journey through all four domains. In the care
processes domain, nurses reported that the requirement to evidence care given by
recording each individual care process element caused them some frustration as it
interrupted care delivery and restricted professional judgement. Where a nurse was
allocated six patients, who each required two care processes ( four hourly SKIN
bundle -14 elements and NEWS bundle - six elements), equated to recording
720 individual elements every 24 hours. We found that nurses ’ interpretation and
recording individual bundle elements varied and with each additional care bundle
variation became greater. Given that the tool for auditing care processes was based on the
care bundle application, it may be timely to scope and address the challenges raised in
applying the care bundles in practice. A care bundle itself does not improve patient safety.
Rather, improvement is generated from re-organising work activity and better
 Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

Figure
Ward A Safe and Effective Process Indicators Safe and Effective Outcome Indicators
Free from Free from
Free from Free from
Patient Critical cardiac arrest hospital
Gender Ag MUST NEWS SKIN FALLS weight loss fall or
Code Medicines or unplanned acquired
e >5% body injurious
admission to pressure
weight fall
ICU ulcer
01A Female 85 0 1 18 ABDF B Low risk Yes Yes Yes
02A Female 88 2 3 18 No triggers B No weight loss Yes Yes Yes

03A Female 82 2 1 18 ACD B,J No weight loss Yes Yes Yes

04A Female 91 0 0 18 A B,E Low risk Yes Yes Yes

05A Female 81 0 0 18 A B,O Low risk Yes Yes Yes

06A Female 73 0 0 17 C B Low risk Yes Yes Yes

07A Female 79 2 2 19 No triggers B No weight loss Yes Yes Yes

08A Female 82 2 3 21 ACD B,J Weight loss Yes Yes Yes

09A Female 89 2 0 19 F B,S No weight loss Yes Yes Yes

10A Male 75 1 4 19 Not completed B No weight loss Yes Yes Yes

11A Female 94 0 2 19 No triggers B,S Low risk Yes Yes Yes Positive Positive Yes Yes
12A Female 88 0 0 21 No triggers B Low risk Yes Yes Yes
13A Male 77 0 2 17 C B,G,O,S Low risk Yes Yes Fall Neutral Nil return Nil return
 staff/patient communication, which improves patient outcomes. Unchecked, audit outcomes on NQI
care processes will therefore be unreliable in providing assurance to the board that safe and framework tool
effective care processes are being applied consistently across the organisation.
Findings also highlighted nurses’ concerns about paper-based documentation for
recording care processes. Nurse recording is discussed extensively in the professional
literature (Hutchinson and Sharples, 2006; Powell, 2006; Griffiths et al., 2007; Muller-Staub et
al., 2007) and in our study, nurses consistently expressed the view that better mechanisms for
recording would improve nursing care. However, Urquhart et al. (2009) concluded that there 613
was no evidence that changing record systems made any difference to nursing practice or
patient outcomes. McCormack et al. (2015) felt that pre-determined elements within
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

electronic records were not conducive to evaluating person-centred nursing care. We

highlighted a need for clarity on exactly what care nurses are required to document, the
recording’ s purpose and how recording can be measured and reported.
Despite care processes and recording issues, we found that patient outcomes were good.
Measuring patient outcomes alongside process measures supports the framework as it adds to a
more rounded view of nursing care quality. Where outcomes are found to be consistently
good, consideration should be given to stepping down routine recording care processes and
diverting resources to development and learning in areas where practice is deemed to be
poor/less than satisfactory.
Gathering the patient’ s experience is key to measuring nursing quality (Griffiths et a l., 2008) as
it identifies strengths and risks to safe and effective care. Patients in our study rep orted
positive nursing experiences, specifically they felt safe, had confidence in the nurses ’
knowledge and skills and that nurses understood what was important to them, which
reflects McHugh and Stimpfel’ s (2012) findings that quality can be measured through softer
indicators such as patient satisfaction with care and hard data on mortality and morbidity
rates. Selective framework domains is also supported by Doyle et al.’ s (2013) theory on the
associations between patient experience, clinical effectiveness and safety, and provides a
mechanism for identifying care improvements. We highlighted a potential safety issue where
several patients reported that they did not want to bother nurses:
I don’t like to annoy them - I try not to drink a lot so I don ’t have to go to the toilet and bother the
nurses. They are very busy (03A).
This patient’ s perception may have unintentionally put him/her at risk of not receiving
necessary care and highlights his/her anxiety not to add to the ward challenges on issues such
staffing levels and time to care. If patients are to be true partners in care design and delivery,
then nurses must act as patient advocates, highlighting issues and being part of the solution.
Assessing the nurses’ knowledge domain, we found that nursing care plans were not
always updated to reflect the patients’ current/on-going needs. Acute nursing care is largely
a continuous activity and nurses reported that paper-based documents did not lend itself to
the recording on-going care over 24 hours. Nevertheless, nurses could describe specific
nursing care needs and patient status at a point in time. All interactions between patient and
staff have the potential to enhance patient care. Engaging frontline staff in quality
improvement initiatives improves care delivery experience, increases job satisfaction and
has a reciprocal positive impact on patient experience and outcomes (National Nursing
Research Unit, 2013).
Our multifarious study employed several data collection methods reflecting the
complexity of synthesising all activities necessary to generate a comprehensive nursing care
quality assessment (Burston et al., 2013; Needleman et al., 2009; Doran et al., 2006).
We generated significant information on four domains and tested the framework’ s
application, indicating what was valuable in providing comprehensive assurance to the
hospital board and information on which to base nursing care improvements.
 IJHCQA The sample size in this research was small and consideration should be given to larger
30,7 scale organisational data collection methods applied across the framework’ s four domains.
The research method was not specifically designed to secure rigorous statistical analysis,
however, there is potential for further analysis using stratification and risk adjustment to
benchmark across different care settings. Further research could also be directed at
understanding process-outcome relationships relevant to safe, effective, person-centred
614 nursing care.

Conclusion
The literature suggests that, given the nursing p rofession’ s complexities, no single measure
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

can provide a complete nursing quality p icture. This study proposed a more comprehensive
means of assuring safe, effective, person-centred nursing care by extending the reporting
elements beyond the singular compliance with care process measurement. Our study found
that care experience is important to patients and ensuring a good care experience lies
primarily with the nurse whose knowledge and skills are essential in shaping person-centred
care ( M cC ormack and M cC ance, 2010). A challenge to performance-driven organisations is
to give assurance reporting a more person-centred focus. We anticipate that a pre-requisite to
applying this framework is having a co-ordinated strategy for improving patient safety and
patient experience in place. We used multiple sources and methods to evidence nu rsing
quality and provide information on which to base improvements. Action planning to effect
change will be monitored with regular updates locally and to the hospital board on
improvements and developments. Patient safety is a central focus in board bus iness. Outputs
from this framework can indicate declining standards and will influence professional practice
developments, and provide nursing teams with an opportunity for reflection and learning.
Although initially designed and tested within the nursing pr ofessions, The framework can be
applied as a mechanism for reporting assurance in other health and social care disciplines, and
externally, such as, regulators and other public bodies.

References
Aiken, L.H., Clarke, S.P. and Sloane, D.M. (2002), “Hospital staffing, organization, and quality of care:
cross national findings”, International Journal for Quality in Health Care, Vol. 14 No. 1, pp. 5-13.
American Nurses Association (2015), Code of Ethics for Nurses with Interpretive Statement, ANA,
Silver Spring, MD.
Aydin, C.E ., Bolton, L.B., Donaldson, N ., Brown, D .S. and M ukerji, A . (2008), “Beyond nursing quality
measurement: the nation’s first regional nursing virtual dashb oard”, in Henriksen, K., Battles, J.B.,
Keyes, M.A. and Grady, M .L . ( Eds) , Advances in Patient Safety: New D irections and Al ternative
Approaches (V ol. 1: Assessment), Agency for Healthcare Research and Quality, Rockville, M D,
pp. 1-18.
BAPEN (2011), Malnutrition Universal Screening Tool, BAPEN, Worcestershire, available at: www.
bapen.org.uk/pdfs/must/must_full.pdf (accessed 6 April 2015).
Blegen, M.A. and Vaughn, T. (1998), “A multisite study of nurse staffing and patient occurrences”,
Nursing Economics, Vol. 16 No. 4, pp. 196-203.
Burston, S., Chaboyer, W . and G illespie, B . (20 13), “Nurse-sensitive indicators suitable to reflect nursing
care quality: a review and discussion of issues ”, Journal of Clinical Nursing, Vol. 23 Nos 13-14,
pp. 1785-1795.
Coyle, Y.M. and Battles, J.B. (1999), “Using antecedents of medical care to develop valid quality of care
measures”, International Journal for Quality in Health Care, Vol. 11 No. 1, pp. 5-12.
Department of Health, Social Services and Public Safety (2011), Quality 2020 - A 10-Year Strategy to
Protect and ImproveQualityin Health and Social Care in Northern Ireland, DHSSPS, Belfast.
De Silva, D. (2014), Helping Measure Person-Centred Care, The Health Foundation, London.
 Donabedian, A. (1988), “The quality of care.How can it be assessed?”, Journal of the AmericanMedical NQI
Association, Vol. 260 No.12, pp. 1743-1748. framework tool
Doran, D., Harrison, M.B., Laschinger, H., Hirdes, J., Rukholm, E., Sidani, S., Hall, L.M., Tourangeau, A.E.
and C ranley, L. (2006), “R elation between nursing interventions and outcome achievement in acute
care settings”, Research in Nursing and Health, V ol. 29 N o. 1, pp. 61- 67.
and Bell, D. (2013), “Asystematic review of evidence on the links between patient
Doyle, C., Lennox, L.
experience and clinical safety and effectiveness”, British Medical Journal, Vol. 3 No. 1, pp. 1-18, 615
doi: 10.1136/bmjopen-2012-001570.
Francis, R. (2013), Report of the Mid Staffordshire NHS Foundation Trust Public Inquiry: Executive
Summary, Stationery Office, London, ISBN: 9780102981476.
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

Gallagher, R.M . and Rowell, P.A. (2003), “Claiming the future of nursing through nursing-sensitive
quality indicators”, Nursing Administration Quarterly, Vol. 27 No. 4, pp. 273-284.
Gibbons, W ., Shanks, H.T., Kleinhelter, P. and Jones, P. (2006), “Eliminating facility acquired pressure
ulcers at ascension health ”, Journal on Quality and Patient Safety, Vol. 32 No. 9, pp. 488-496.
Griffiths, P., Debbage, S. and Smith, A. (2007), “A comprehensive audit of nursing record keeping
practice”, British Journal of Nursing, Vol. 16 No. 21, pp. 1324-1327.
Griffiths, P., Jones, S., Maben, J. and Murrells, T. (2008), State of the Art Metrics for Nursing: A Rapid
Appraisal, National Nursing Research Unit, King’s College, London.
Heslop, L. and Lu, S. (2014), “Nursing sensitive indicators: a concept analysis ”, Journal of Advanced
Nursing, Vol. 70 No.11,pp. 2469-2482.
Hutchinson, C. and Sharples, C. (2006), “Information governance: practical implications for record-keeping”,
Nursing Standard, V ol. 20 N o. 36, pp. 59-64.
Lincoln, Y.S. and Guba, E.G. (1985), Naturalistic Inquiry, Sage Publications, Beverly Hills, CA.
McCance, T., Telford, L., Wilson, J. and Macleod, O. (2012), “Identifying key performance indicators for
nursing and midwifery care using a consensus approach ”, Journal of Clinical Nursing, Vol. 21
Nos 7-8, pp. 1145-1154.
McCormack, B. and McCance, T. (2010), Person-Centred Nursing: Theory and Practice, Wiley-
Blackwell, London.
McCormack, B., Borg, M ., C ardiff, S ., Dewing, J ., Jacobs, G ., Janes, N ., Karlsson, B., M cC ance, T.,
Mekki, T.E., Porock, D., van Leishout, F. and Wilson, V. (2015), “Person-centredness - the ‘state’ of
the art”, International Practice Devel opment Journal, V ol. 5 (Special issue on person-centredness,
Article 1), pp. 1-15, available at: www.fons.org/library/journal.aspx ( accessed 15 September 2015 ).
McGillis Hall, L., Doran, D. and Pink, G. (2004), “Nurse staffing models, nursing hours, and patient
safety outcomes”, Journal of Nursing Administration , Vol. 34 No.1, pp. 41-45.
McHugh, M.D. and Stimpfel, A.W. (2012), “Nurse reported quality of care: a measure of hospital
quality”, Research in Nursing and Health, Vol. 35 No. 6, pp. 566-575.
and Griffiths, P. (2012), High Quality Care Metrics for Nursing,
Maben, J., Morrow, E., Ball, J., Robert, G.
National Nursing Research Unit, King’ s College, London.
Montalvo, I. (2007), “The national database of nursing quality indicators TM (NDNQI®)”, OJIN:
The Online Journal of Issues in Nursing, Vol. 12 No. 3, pp. 1-8.
Muller-Staub, M., Needham, I., Odenbreit, M., Lavin, M.A. and van Achterberg, T. (2007), “Improved
quality of nursing documentation: results of a nursing diagnoses, interventions and outcomes
study”, International Journal of Nursing Terminologies and Classifications, Vol. 18 No. 1, pp. 5-17.
National Nursing Research Unit (2013), Policy+: D oes NHS Staff W ellbeing Affect Patient Experience of
Care?, No.39, King’s College, London.
National Patient Safety Agency (2010), “Reducing harm from omitted and delayed medicines in
hospital”, Rapid Response Alert, NPSA/2010/RRR009, available at: www.nrls.npsa.nhs.uk
(accessed 6 April 2015).
IJHCQA
30,7

616
Needleman, J., Buerhaus, P., Mattke, S., Stewart, M. and Zelevinsky, K.I. (2002), “Nurse-staffing levels and the quality of care in hospitals”, New
England Journal of Medicine, Vol. 346 No. 22, pp. 1715-1722.
Needleman, J., Parkerton, P., Pearson, M., Soban, L., Upenieks, V.and Yee, T. (2009), “Overall effectof
TCAB on initial participating hospitals ”, American Journal of Nursing, Vol. 109, pp. 59-65.
NHS Quality Improvement Scotland (2005), Clinical Governance: Achieving Safe, Effective, Patient- Focused
Care and Services, NHS QIS, Edinburgh , available at: www.nhs.healthquality.org (accessed
23 March 2015).
NHS Wales (2010), “Preventing hospital acquired pressure ulcers”, available at: www.1000livesplus. wales.nhs.uk (accessed 6 April 2015).
Northern Ireland Practice Education Council (2011), “Evidencing care through key performance indicators for nursing and midwifery” , available at:
www.nipec.hscni.net/cw_evidencingcare.html (accessed 23 March 2015).
Nursing and Midwifery Council (2015), The Code, Professional Standards of Practice and Behaviour for
Nurses and M id w ives, N M C , London.
Perla, R.J., Provost, L.P. and Murray, S.K. (2013), “Sampling considerations in health care improvement”, Quality Management in Health Care,
Vol. 22 No.1, pp. 36-47.
Powell, J.(2006), “Sharingelectronic health records”, Informatics in Primary Care, V ol. 14 N o. 1, pp. 55-57. Resar, R., Griffin., F.A., Haraden, C. and Nolan, T.W.
(2012), “Using care bundles to improvehealth care
quality”, IHI Innovation Series white paper, Institute for Healthcare Improvement Cambridge,
MA, available at: www.IHI.org (accessed 6 April 2015).
Royal College of Physicians (R C P) (20 12a), National Early Warning Score (NEWS) - Standardising the Assessment at Acute Illness Severity in the NHS, RCP,
London, available at: www.rcplondon.ac. uk/projects/outputs/national-early-warning-score-news (accessed 6 April 2015).
Royal College of Physicians (RCP) (2012b), FallSafe Care Bundles, RCP, London, available at:
www.rcplondon.ac.uk/guidelines-policy/fallsafe-resources-original (accessed 6 April 2015).
Savitz, L .A ., Jo nes, C .B . and Bernard, S. (2005), “Quality indicators sensitive to nurse staffing in acute care settings ”, in Henriksen, K., Battles, J.B., Marks,
E.S. and Lewin, D.I. (Eds), Advances in Patient Safety: From Research to Implementation’ , V ol. 4, Programs, Tools, and Products, AHRQ Publication
02-0021-4, Agencyfor HealthcareResearchand Quality, Rockville, MD, pp. 375-385.
Urquhart, C ., Currell, R., Grant, M.J. and Hardiker, N.R. (2009), “Nurse record systems: effects on nursing practice and healthcare outcomes”,
Cochrane Database of Systematic R eview s, No. 1, pp. 1-69 (Art. No.CD002099).
US Institute of Medicine (2001), Crossing the Quality Chasm: A New Health System for the 21st Century, National Academy Press, Washington, DC.
Welsh Assembly Government (2010), All Wales Nursing Dashboard, Welsh Assembly Government, Cardiff.

Corresponding author
Dawn Connolly can be contacted at: dawn.connolly@southerntrust.hscni.net

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/328033441

Identifyingvalidatednursing qualityindicators forthe intensive care unit: an

integrative review
Article · A ugust 2018
DOI: 10.1891/1748-6254.12.2.28

CITATIONS READS

0 812

4 authors:

Efi Evangelou Ekaterini Lambrinou

Cyprus University of Technology Cyprus University of Technology
1 PUBLICATION 0 CITA TIONS 113 PUBLICATIONS 1,837 CITA TIONS

SEE PROFILE

Christiana K outa
 Cyprus University of Technology SEE PROFILE

93 PUBLICATIONS 440 CITA TIONS

Nicos Middleton
Cyprus University of Technology
184 PUBLICATIONS 2,104 CITA TIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

IENE6 European Project View project

WoMEnpower (WE-ME) Project View project

All content following this page was uploaded by Ekaterini Lambrinou on 22 October 2018.

The user has requested enhancement of the downloaded file.

 vvRCELSIN I RACLHCCOONNNNEECCT
IONNSS I v
Identifying validated nursing quality
indicators for the intensive care unit: an integrative review
*Efi Evangelou RN; BSc, MSc, Research Associate, Department of Nursing, School of Health Sciences, Cyprus
University of Technology, Limassol, Cyprus
Ekaterini Lambrinou RN; BSc, MSc, PhD, Associate Professor, Department of Nursing, School of Health Sciences, Cyprus
University of Technology, Limassol, Cyprus
Christiana Kouta RN; BSc, MSc, PhD, Associate Professor, Department of Nursing, School of Health Sciences, Cyprus University of
Technology, Limassol, Cyprus
Nicos Middleton BSc, MSc, PhD, Associate Professor, Department of Nursing, School of Health Sciences, Cyprus University of
Technology, Limassol, Cyprus
Key words: AIRE instrument vintensive care units vnurse staffing vnursing care vpatient outcomes vquality indicators v
*Email: ekaterini.lambrinou@cut.ac.cy

SUMMARY
• Background: Quality indicators (QIs) play an important role in
evaluating quality improvement initiatives. A generally accepted
set of QIs specific to the nursing care in the intensive care unit
(ICU) is not available.
• Aim: To identify QIs associated with nursing care for adult
ICU in the literature. The methodological quality of QIs was
assessed and associated variables of quality and quantity of
nursing care were also identified.
• Methods: We employed an integrative literature review. A
focused search of electronic databases was applied. Inclusion
and exclusion criteria were used for the selection of relevant
articles. Quality assessment of the included studies was based
on the guidance document of domains and elements suggested
by the Agency of Health Care Research and Quality. QIs’
methodological quality was assessed using the Appraisal of
Indicators through Research and Evaluation (AIRE) instrument.
• Results: The review identified 13 studies and 45 QIs associated
with nursing care in the ICU. The set of QIs assessed in each
study, the type of nurse staffing measure as well as morbidity
and mortality rates varied considerably. Findings suggest that
quality and quantity of nursing care are strongly associated with
higher rates of adverse events, mortality, infections and
complications in adult ICUs. Methodological quality of the
QIs also differed considerably. Higher AIRE scores, indicating
higher scientific evidence of methodological quality, can be
used to select evidence-based and valid QIs.
• Conclusions: A number of QIs quantifying nursing care in the
ICU have been identified. These QIs could be combined to form
a tool which would allow to the quantification and assessment
of the quality of ICU nursing care provided in a regular basis.
INTRODUCTION
financial’ resources (Robert et al., 2000). In the ICU, adverse events
and human error constitute substantial morbidity and mortality risks
to critically ill patients; especially as a result of understaffing (De Vos
et al., 2007).
Quality indicators (QIs) play an important role in quality improvement
initiatives as long as they are based on evidence (Brook et al., 1996).
Nursing care is considered a critical factor of patient care and thus,
the assessment of QIs which specifically reflect nurses’ contribution
is vital (Montalvo, 2007). The identification of QIs that quantify
the quality of nursing care would enable the development of sets
that could be easily adopted in practice in the context of auditing
performance and monitoring evidence-based practice in routine care
(Mainz, 2003).
In essence, QIs are screening tools, which identify potential
suboptimal clinical care and reveal specific problematic areas
that need further investigation (De Vos et al., 2007). QIs may be
classified according to three dimensions, which correspond to
structure, process and outcome components of quality (Donabedian,
1992). Outcome indicators, in particular, reflect the effect of quality
of patients’ health care (Mainz, 2003). Thus, potential QIs of nursing
care quality may be identified by the extent to which they indicate
nurse sensitive outcomes (NSOs). NSOs refer to aspects of patients’
experience, behavior or health clinical status, which are completely
or partially determined by the quantity and quality of nursing care
received. These may include outcomes, which are influenced by
several factors, as long as there is evidence to suggest that they
are also associated with nursing care (Montalvo, 2007). Nursing QIs
are specific to nursing and may thus differ from medical indicators
of care quality. They have been defined as those which are based
on “nurses’ scope and domain of practice, and for which there is
empirical evidence linking nursing inputs and interventions to the
outcome” (Doran and Pringle 2003, vii). It is important for nursing
QIs to form a valid and reliable means of assess nursing care quality
(Heslop and Lu 2014). The QIs in the ICU should be relevant to the
problem, understandable, measurable, behaviourable (and thus,
amenable to change), achievable and feasible, according to the
RUMBA rule (Braun et al., 2010).

Care on intensive care units (ICUs) represents a major portion of Most of the published studies examining the relationship between
nurse staffing and patient outcomes have been performed in
health care costs and thus consumes a large part of a hospital's
}

2 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
general wards or at the entire hospital level (Griffiths et al., 2008,
Needleman et al., 2002, Van den Heede et al., 2007). National
forums and agencies have proposed a number of QIs. Nevertheless,
specific QIs for the ICU have not been suggested (AHRQ, 2006,
ΑΝΑ, 2000, JCAHO, 2007, NQF, 2004). Furthermore, forums and
societies specific to the ICU propose QIs, but none is specifically and
exclusively referred to as nursing-related QIs (Delgado et al., 2005,
Ray et al., 2009). Currently, there is not a complete set of QIs specific
to the nursing care in the ICU available (McGahan et al., 2012). In
an effort to fill the gap, this study reviews the literature and pools
potential QIs associated with nursing care in the ICU. These QIs may
be used for the development of nursing QI sets (NQI) specific to
the ICU, currently missing from the scientific literature. Identifying
a select set of evidence-based and valid NQI offers the potential to
assess nursing care delivered in the ICU.
AIM
To review the literature in order to identify potential Qis, specifically
patient-centered clinical NSOs, that may be measured in the ICU
and have been found to be associated with variables reflecting the
quantity and/or quality of nursing care (i.e. nursing and setting
structure variables). The secondary aim was the assessment of the
methodological quality of the QIs identified.
METHODS
Design
This is an integrative review of published literature based on a
modified framework for review research (Whittemore and Knafl
2005). The process for data extraction and synthesis was based on a
detailed pre-specified protocol, as presented in the sections to follow.
Search methods
The process of searching the literature was performed based on the
Center of Review and Dissemination guidance (CRD, 2009). A
focused search of electronic databases Ovid Medline, PubMed,
Cumulative Index of Nursing and Allied Health Literature (CINAHL)
and Cochrane library was applied to identify the relevant literature.
The time frame of interest was 2000-2016.
The search was undertaken by using the following keywords and
medical subject heading (Mesh) terms: “intensive care unit(s)”
[Mesh] OR “ICU”, “quality indicators- health care” [Mesh] OR “clinical
indicators”, and nurs* (nurses, nursing, nurse staffing, personnel
staffing, critical care nursing) [Mesh]. The search terms were used
in all possible combinations using Boolean operators. The “similar
articles” tool of PubMed was also used. Furthermore, the reference
lists from included studies were also reviewed in order to identify
additional studies that may have not been identified in the original
search. Authors did not attempt any hand searching of journals,
conferences’ or abstracts’ proceedings.
Variables of interest referred to the education and certification level,
years of experience, hours of nursing care per patient day, workload
intensity and overtime. Nurse staffing measures such as staff mix, skill
mix, staff ratio, and nurse to patient (N/P) ratio were also considered
of interest. Nurse staffing measures are already recognized as
structure QIs. In each one of the selected studies these measures
were examined for possible association with patients’ outcomes. Any
ICU patient-related clinical outcome was extracted and considered
as a potential QI.
Specific inclusion criteria were applied. Only published research
articles were selected. Studies were included if they had examined
associations between nursing and setting related structure variables
with critically ill adult patients’ outcomes. Studies that evaluated QIs
29
already recognized as NSOs were also included. Secondary data
analysis surveys were also included. Only quantitative studies were
identified and included. For candidate QIs in any of the identified
studies, the numerator and denominator of QIs should be given or
at least it should be easily understood from the description. The
relativeness to any type of ICU was set as an additional selection
criterion.
Articles published in language other than English, unpublished
studies, abstracts, editorials, expert opinion papers, secondary
sources, such as reviews or systematic literature reviews, conceptual
sources, anecdotal and opinion sources were excluded. Studies
that assessed the association of QIs with nurse staffing outside the
hospital, after discharge from the ICU, at the ward or during transfer
from the ICU, were excluded. Studies that presented results at a
hospital level were also excluded. Other structure components of
quality and organizational characteristics such as daily rounds and
team work were not considered.
Quality appraisal
The methodological quality of the included studies was independently
assessed by two of the authors and was based on the domains and
elements suggested by the Agency of Health Care Research and
Quality. The assessment considered: study question, population,
comparability of subjects, exposure or intervention, outcome,
statistical analysis, results, discussion, funding or sponsorship (West
et al., 2002).
Additionally, the methodological quality of the included QIs was
assessed using the Appraisal of Indicators through Research and
Evaluation (AIRE) instrument. The AIRE instrument is a new tool,
which was designed and validated in the Netherlands (De Roo et
al., 2013). It was previously used in peer-reviewed studies aiming
to develop a set of QIs for palliative care (De Roo et al., 2013), for
midwifery care (De Bruin-Kooistra et al., 2012), of musculoskeletal
injury management (Strudwick et al., 2015) and care of osteoarthritis
(Petrosyan et al., 2017). AIRE addresses the face and construct
validity, accuracy, risk of bias, ability to achieve real quality
improvement, and application of QIs (Strudwick et al., 2015).
In this review, the AIRE instrument was used to assess whether the
objective and the organizational background of the identified QIs are
well defined and the extent to which they built on evidence (Smeulers
et al., 2015). AIRE contains four domains that examine: 1. Purpose,
relevance and organizational context of the QIs, 2. Stakeholders’
involvement for the development of the QIs, 3. Scientific evidence
and 4. Additional evidence, formulation, usage. Additionally, there are
in total twenty items (e.g. “systematic methods were used for search
for scientific evidence”) across these four domains. Each of the four
authors independently scored on a 4 point Likert scale (1= strongly
disagree and 4= agree) for the items given (De Roo et al., 2013).
The maximum score for an item is 16 and the minimum is 4, when
four authors are asked to rate. In the present study AIRE was applied
for the set of the QIs identified in each study, rather than for each
QI separately (Smeulers et al., 2015). In the absence of guidance
regarding definition of high, medium and low scores for the items,
the authors considered as high: 16-14, medium: 13-9 and low: 4-8.
Higher scores are indicating valid and widely used QIs. A score of
50% and higher in all four domains indicates higher methodological
quality (Strudwick et al., 2015).
Data extraction
A structured data extraction form was used to collect information from
the studies, which are summarized in Table 1. The data extraction
form included: 1. Authors/ publication date, Country and Data
collection period, 2. Research aim(s), 3. Methods, and 4. Quality
indicators and formula or definition 5. Nurse staffing variables and
2018 Volume 12 Number 2 The World of Critical Care Nursing
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
definition and 6. Results (main). The initial selection included the
screening of titles and abstracts against the inclusion criteria. The
second stage is referred to the screening of the full papers in order
to identify articles that fulfill the inclusion criteria. Two of the authors
extracted information and independently reviewed eligibility criteria
of the articles obtained. Any disagreements were resolved through
consensus, and when necessary, with the involvement of the third
author. The process ended up with full consensus after detailed
examination of full text articles and consideration of predetermined
inclusion criteria. In the case of incomplete information the article
was excluded. A flow diagram of the search strategy is depicted in
Figure 1.
Data synthesis
Similar data were categorized and organized together so as to
enable comparison and interpretation (Table 1). Moreover, QIs
were categorized as primary (e.g. mortality, cost) and secondary
(e.g. infections, complications) (Amavardi et al., 2000, Pronovost
et al., 1999) and further classified based on a previously described
framework (Doran and Pringle 2011, Holzemer 1994). The
subcategories included: setting- related outcome variables (QIs) and
patient safety related outcome (Heslop and Lu 2014), as shown in
Table 2.
RESULTS
Search outcomes
The search yielded 83 article titles of which only five articles
were selected for further analysis based on the methodological
assessment. Full text was obtained for all five articles and eight
additional studies were identified, either from the reference lists, or
classified by the database as related articles to those already
obtained. A total of 13 studies were included in this review (Figure 1).
The studies refer to 45 QIs in total (Table 1 and 2).
Overview of the included studies
There is a growing interest worldwide regarding patients’ outcomes
that are affected by the nursing care provided. The included studies
originated from Asia (Cho et al., 2008), Switzerland (Bracco et al.,
2001), Scotland (Tarnow- Mordi et al., 2000), Austria (Metnitz et al.,
2008), one was a multi-centered worldwide study (Valentin et al.,
2006) and the remaining eight studies originated from the USA.
Across the selected studies, the respective ICUs differed in type
and included surgical, medical and mixed ICUs. Τhe number of
participating ICUs ranged from eight (Alonso- Echanove et al., 2003)
to as many as 205 (Valentin et al., 2006). The number of involved
hospitals and their capacity also differed, while, as shown in Table
1, a number of studies did not report the type or number of included
hospitals (Kendall-Gallagher and Blegen, 2009, Metnitz et al., 2008,
Valentin et al., 2006).
Quality appraisal
All included studies used an observational design. Six studies used
prospective data (Alonso- Echanove et al., 2003, Bracco et al., 2001,
Garcia and Fugulin, 2012, Metnitz et al., 2008, Robert et al., 2000,
Valentin et al., 2006) and seven studies collected retrospective data
(Amavardi et al., 2000, Cho et al., 2008, Dang et al., 2002, Kendall-
Gallagher and Blegen, 2009, Stone et al., 2007, Tarnow-Mordi et al.,
2000, Whitman et al., 2002). Data collection methods included: 1.
Questionnaires regarding ICU and/or hospital characteristics which
were completed by the medical director of the unit (Amavardi et al.,
2000, Dang et al., 2002, Metnitz et al., 2008) or the head nurse of
the unit (Tarnow-Mordi et al., 2000) of ICUs, 2. Forms specifically
designed by the research teams which were completed by MDs and
nurses (Alonso- Echanove et al., 2003, Bracco et al., 2001, Robert
2 2018 Volume 12 Number 2
et al., 2000, Valentin et al., 2006) or infection control practitioners
(Alonso- Echanove et al., 2003). One study (Whitman et al., 2002)
used a combination of qualitative and quantitative data collection
through interviews and special forms (Table 1).
All of the studies included a focused aim or a research question
(Table 1). Only three studies used QIs already recognized as nursing
specific (Bracco et al., 2001, Valentin et al., 2006, Whitman et
al., 2002). The remaining ten studies clearly aimed to explore the
association between structural variables and patient outcome(s).
Sample sizes of the included studies ranged from 28 (Robert et al.,
2000) to 83259 (Metnitz et al., 2008) patients. Two studies do not
clearly report specific inclusion and exclusion criteria for the sample
(Garcia and Fugulin, 2012, Kendall-Gallagher and Blegen, 2009).
The majority of the sample employed multivariable analyses (Alonso
-Echanove et al., 2003, Amavardi et al., 2000, Bracco et al., 2001,
Kendall-Gallagher and Blegen, 2009, Metnitz et al., 2008, Stone et
al., 2007, Tarnow-Mordi et al., 2000).
Five studies report funding (Cho et al. 2008, Kendall-Gallagher and
Blegen, 2009, Stone et al., 2007, Valentin et al., 2006, Whitman et
al., 2002), whereas ethical approvals were sought and granted in six of
the selected studies (Amavardi et al., 2000, Bracco et al., 2001,
Garcia and Fugulin, 2012, Kendall-Gallagher and Blegen, 2009,
Metnitz et al., 2009, Whitman et al., 2002).
Table 3 shows the scores of the QIs based on AIRE (De Roo et al.,
2013). The highest scores were obtained for items 18 (97%) and
10 (96%) indicating that the QIs had been tested in daily practice
and were evidence-based, respectively. Most of the sets achieved
the highest scores (89%) for category III “Scientific evidence” and
the lowest (51%) for the category II “Stakeholder involvement”
(Smeulers et al., 2015). Only Dang et al. (2002) and Amavardi et al.
(2000) reported the development process and recruited a panel to
identify ICD-9-CM codes for the QIs. The remaining eleven studies
applied QIs that have been widely used in previous studies. Αll of the
included studies provide the definitions or the formulas for calculation
of QIs. None of the included studies provided full description of
terminology, rationality or justification, source of the data and type
of the parameter.
Quality indicators identified
Findings of the included studies regarding rate measures of QIs and
main associations identified in each study are presented in Table 1.
The QIs identified were grouped and classified in subgroups (Table
2). The domain that appears to be mostly covered is safety (Doran
and Pringle, 2011). Negative performance QIs were most common,
such as adverse events, infections and complications (Mitchell,
2008). Most commonly used QIs are mortality and blood stream
infections (BSIs).
There are QIs with a variety of names that examine the same
numerators and denominators such as decubitus ulcers (Stone et al.,
2007) and skin break down (Kendall- Gallagher and Blegen, 2009).
However, in the case that different numerators and denominators are
used for seemingly identical QIs, these were regarded as different
outcomes. Likewise, there are similar QIs that may be considered
part of a more general QI, such as device related catheter-associated
urinary tract infection (CAUTI) and urinary tract infection (UTI).
Identified definitions are provided in Table 1.
Valentin et al. (2006) and Bracco et al. (2001) used sentinel QIs
(undesirable events that trigger further investigation) whereas the
remaining studies applied rate-based QIs. In some studies, QIs were
restricted to specific patient groups of the ICU. Robert et al. (2000)
and Alonso-Echanove et al. (2003) focused on patients with Central
Venous Catheter (CVC). Amavardi et al. (2000) examined patients
with esophageal resection and Dang et al. (2002) was interested in
patients who underwent abdominal aortic surgery. Stone et al. (2007)
focused on elderly patients.
}
The World of Critical Care Nursing
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Authors, Research aim(s) Methods: study
publication date, design, population,
country tools used study
duration
Kendall-Gallagher & To explore: • Correlational, cross-
Blegen, 2009, USA • the relationship sectional, unit level
between the design,
proportion of • Secondary data
certified staff analysis of 48
nurses and the adult ICUs from 29
risk of harm of hospitals randomly
patients selected
• the organizational • Quarterly collected
and nursing data using two
characteristics questionnaires
associated with • One year study
rates of adverse
events
Cho et al., 2008, To examine the • Correlational study
Korea relationship between collecting data
nurse staffing and patient from administrative
mortality in ICUs databases
• 27,372 ICU patients
discharged from
42 tertiary and 194
secondary hospitals
(total 236 hospitals)
aged > 15 years
old with 26 primary
diagnoses
• 3 data sources were
used: ICU survey
data (hospital and
ICU characteristics);
Medical claims data
(patients clinical
and utilization
information); and
NHI (enrollee
database for death
day)
• 3 months
Robert et al., 2000, To determine the risk • Nested case control
USA factors for primary BSI, study
including the effect of • 28 patients with
RNs’ levels primary BSI (case)
compared with 99
randomly selected
patients (control)
hospitalized for
more than 3 days of
a 20-bed Surgical
ICU in a 1000 bed
university- affiliated
inner city teaching
public hospital
• A standardized form
was used for data
collection, as well
as the relative
microbiological data
• 1-year study period
Dang et al., 2002, To isolate the effects of • Retrospective study
USA nurse staffing on with secondary
patient’s outcomes analysis of hospital
(medical complications discharge data
associated with mortality • 2606 patients aged
and are nurse sensitive) 30 years or older
by examining the with principal
association between procedure code for
ICU nurse staffing and abdominal aortic
the likelihood of medical surgery,
complications for patient • 38 ICUs in one state
undergoing abdominal • Data were obtained
aortic surgery by: Health discharge
data set was
used for patient
information; A
questionnaire was
used for nurse
staffing data; An
instrument of 32
items was used for
the organizational
characteristics;
and a panel of
experts including
4 physicians who
identified the
ICD-9-CM codes
for the potential
complications
Table 1. Characteristics of the included studies
31
QIs and formula or
definition identified
• Falls: annual rate per
1000 patient days
• Medication
administration errors
(MAEs): annual rate per
1000 patient days
• Skin break down:
annual rate per 1000
patient days
• Urinary tract infection
(UTI): annual rate per
1000 patient days
• Central catheter line
infection (CCLI): annual
rate per 1000 patient
days
• Bloodstream infection
(BSI): annual rate per
1000 patient days
• Mortality: deaths that
occurred in the hospital
or on the date of
hospital discharge
• Primary BSI: BSI were
defined according to the
CDC criteria per 1000
patient days
• Complications related
to mortality and are
nurse sensitive:
• Cardiac complications:
- acute MI (ICD-9-CM
code: 410); arrest (ICD-
9-CM code: 4275)
• Complications after a
procedure (ICD-9-CM
code: 9971)
• Respiratory
complications:
pulmonary insufficiency
after a procedure
(ICD-9-CM codes:
5184, 5185, 5188);
reintubation (ICD-9-CM
code: 9604); aspiration
ICD-9-CM codes: 507,
9973; ventilation > 96
hrs ICD-9-CM code:
9672
• Others complications:
acute renal failure
ICD-9-CM code: 584;
septicemia: ICD-9-CM
code: 038; platelets
transfusion ICD-9-CM
code: 9905
2018 Volume 12 Number 2
Nurse staffing variables
and definition identified
• Registered nurses
(RNs) education level:
percentage of RNs with
bachelor of science
or higher education in
nursing
• RN years of experience:
mean years of
experience of RNs
• Total hours of nursing
care per day: mean
total working hours of all
nursing staff per day
• RN skill mix: percentage
of nursing staff who are
registered nurses
• RN work group
competence: percentage
of certified staff RNs
• Nurses’ years of work
experience: the years of
RNs’ license to the time
yr of data collection
• Staffing of RNs: ratio of
average daily census to
the total number of full
time equivalent RNs in
ICUs (ADC/RN ratio)
• Composition of nursing
staff (pool versus regular
nurses):
• Regular staff: nurses
permanently assigned
to the unit
• Pool staff: nursing staff
that are members of the
hospital pool service or
agency nurses who work
at the hospital
• Mean of nurse to patient
ratio (N/P) is expressed
as the maximum number
of nursing hours per
SICU-patient day
• Intensity of nursing staff:
the average N/P ratio at
day and night
• Low, medium and high
intensity nurse staffing
at day and night was
calculated
The World of Critical Care Nursing
Results
• Expected rate of falls was 1.1 per 1000 patient days
• Expected rate of UTI was 2.29 per 1000 patient days
• Expected rate of BSI was 1.7 per 1000 patient days
• Expected rate of MAEs was 4.82 per 1000 patient days
• The proportion of certified RNs at the unit was inversely rel
to the rate of falls (p = 0.04)
• UTI decreased by 0.19 (expected rate = 2.29 per 1000 patie
days) for each 1 SD change in the proportion of certified sta
nurses in the unit (p = 0.07)
• The total hours of nursing care was positively related to MA
(p = 0.006).
• Mean years of experience by staff nurses was inversely rel
to UTI (p = 0.01)
• Every additional patient per RN was associated with 9%
increase in the odds of death (OR = 1.09, 95% CI 1.04-1.14
• Nurse education was negatively related to skin break down
(-0.44, p = 0.05)
• Nurse experience was positively related to MAE (0.31, p =
• Total hours of Nursing care to patient day was positively
correlated with CLBSI (0.62, p = 0.01) and catheter infectio
(0.63, p = 0.01)
• Nurse skill mix was correlated positively with MAEs (0.31, p
0.05) and negatively with UTI (0.64, p = 0.05)
• The overall mortality rate was 17% in tertiary and 22% in
secondary hospitals
• There was a greater likelihood of dying to patients admitted
to mixed ICU in tertiary hospitals (OR = 1.61, p = 0.011) an
in hospitals where there was not a board- certified physicia
present for 4 or more hrs per day (OR = 1.56, p = 0.002)
• Patients at secondary public hospitals who located in
metropolitan cities had greater probability of dying (OR = 1
p = 0.005)
• The ADC/RN ratio was significantly related with mortality in
secondary hospitals ( OR = 1.43, 95% confidence interval [
1.16-1.77, p = 0.001)
• Every additional patient per RN in secondary hospitals was
associated with a 9% increase in odds of dying (OR = 1.09
95% CI 1.04-1.14).
• Every additional patient per RN was associated with 9%
increase in the odds of death (OR = 1.09, 95% CI 1.04-1.14
• Overall primary BSI was 4.6 per 1000 patient days
• There was a decreased regular N/P ratio (9.1 hrs/ patient, p
0.001) and an increased pool-nurse-to-patient ratio (4.4 hrs
patient, p < 0.001) during the period of 5 months with decre
regular nurse to patient ratio
• BSI were significantly more frequent during periods with
decreased regular nurse to patient ratio than period with hig
patient ratio (7.6 vs 2.8 BSI/1000 patient days, respectively
= 0.004)
• BSI were significantly more frequent in the period of decre
regular nurse to patient ratio (9.1 hours/ patient, p < 0.001
increased pool-nurse-to patient ratio (4.4 hrs/ patient, p < 0
• Regular N/P ratio for the 3 days before the index date was
significantly lower for case patients than for controls (media
vs 9.9 nursing hours / patient day, p < 0.001)
• Pool N/P ratio was significantly higher for case patients tha
for controls (median 3.2 vs 2.8 nursing hours / patient day,
< 0.001)
• The intensity of nurse staffing was significantly associated
all the complications examined
• Patients treated on units with medium intensity staffing wer
more likely to have cardiac complications (OR = 1.78, 95%
1.16-2.72, p = 0.29) and other complications (OR = 1.74, 9
CI = 1.15-2.63, p = 0.49) comparing with those treated in u
with high intensity
• Patients cared in units with low intensity staffing were more
twice as likely to have respiratory complications than patien
treated in units with high intensity staffing (OR = 2.33, 95%
1.50-3.60, p = 0.14)
• Patients cared in units with medium intensity staffing were
more than twice as likely to have cardiac complications afte
a procedure, than patients treated in units with high intensit
staffing (OR = 2.10, 95% CI 1.26-3.50)
• Patients were more than 5 times as likely to develop pulmo
insufficiency after surgery (OR = 5.11, 95% CI 2.89-9.04), a
well as more than twice as likely to be mechanically ventila
after 96 hours (OR = 2.39, 95% CI 1.55-3.69) and reintuba
(OR = 2.09, CI 1.47- 3.03) on units with low intensity staffin
compared with units with high intensity
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v

Authors, Research aim(s)
publication date,
country
Amavardi et al., 2000, To determine if the
USA presence of N/P ratio
at night time (NNPR) of
one nurse caring for one
or two patients vs one
nurse caring for three or
more patients in the ICU
is associated with clinical
and economic outcomes
following esophageal
resection
Methods: study QIs and formula or Nurse staffing variables Results
design, population, definition identified and definition identified
tools used study
duration
• Statewide • Primary: mortality; • N/P ratio at night shift • The overall unadjusted in hospital mortality rate for esophag
observational cohort hospital LOS; cost (NNPR): resection was 8.1%
study • Secondary: post- • Nurse caring for > 1: 2 • Unadjusted mortality for patients with NNPR < 1: 2 vs NNP
• 353/366 adult ICU operative infection means that 1 nurse is 1: 2 was 15% vs 5.6% (p = 0.009)
surgical patients (ICD-9-CM code:9985); caring for 1 or 2 patients • No significant difference was found n the risk of in hospital
aged 18 years and aspiration (ICD-9-CM • Nurse caring for < 1: mortality between patients with a NNPR > 1: 2 and those w
older in 32/35 acute codes: 507, 9973); 2 means that 1 nurse NNPR < 1: 2 (OR = 0.7, 95%, CI 0.3-2.0)
care hospitals with reintubation (ICD-9-CM is caring for 3 or more • Median LOS for patients with NNPR < 1: 2 vs NNPR > 1: 2
primary procedure code: 9604); pulmonary patients 15 days vs 9 days (p < 0.001).
code of esophageal insufficiency (ICD-9-CM • There was 39% increase in LOS for patients when NNPR <
resection/ codes: 5184, 5185, (95% CI, 19-61%, p < 0.001)
• Data were obtained 5188); pneumonia • Increased LOS was associated with low surgeon volume (p
from: the hospital (ICD-9-CM codes: 0.001), age (p = 0.004) and emergency admission (p < 0.00
discharged 480-487); septicemia • Total hospital cost for patients with NNPR < 1: 2 vs NNPR >
database; a ICD-9-CM code: 038; was $24,915 vs $15,209 (p < 0.001).
previously validated cardiac complications • There was a 32% increase in direct hospital cost ($4810) fo
questionnaire with (ICD-9-CM code: patients with NNPR < 1: 2 (95% CI,14-52%, p < 0.001)
32 organizational 9971); cardiac arrest • Morbidity and resource utilization were increased for patien
characteristics; (ICD-9-CM code: 4257); treated by nurse caring for more than three ICU patients at
ICD-9-CM codes acute MI (ICD-9-CM night.
including primary code: 410); renal failure • - Patients with NNPR < 1: 2 had increased risk of reintubati
diagnosis, 14 (ICD-9-CM code: 584); (OR 2.6, 95% CI, 1.4-4.5, p = 0.001), pneumonia (OR 2.4,
secondary discharge reoperation for bleeding CI 1.2-4.7, p = 0.012) and septicemia (OR 3.6, 95% CI 1.1-
diagnoses and (ICD-9-CM codes: p = 0.04)
14 secondary 3941, 3949, 3998);
procedures; a surgical complications
panel of 2 ICU (ICD-9-CM codes:
physicians for 9981, 9982, 9983)
selected secondary
outcomes that reflect
post- operative
complications/
• 4-year study

Alonso-Echanove et To evaluate the role of
al., 2003, USA the patient, CVC and
nurse staffing factors
in the risk of CVC-
associated BSI
• Prospective, • CVC associated with
observational
multicenter, cohort
study conducted
in a non- research
setting
• 4535 adults
in 8 ICUs at 6
district hospitals
representing 8593
CVCs and 56627
catheter days
• Data collection was
based on NNIS
system and then
transmitted to CDC
• 3 data collection
forms were
developed: patient
admission form;
daily log form; CVC
log form
• Nurse staffing factors:
BSI: according to the float nurse: a nurse
NNIS system definition normally assigned
per 1000 CVC days elsewhere in the hospital
or from an outside
agency, among others
• Median N/P ratio:
number of registered
nurses for each patient
• Median patient care
assistant to patient ratio:
number of patient care
assistants per shift per
100 patients
• 27 of 60 potentially risk factors were significantly associate
with CVC associated BSI (p < 0.05)
• 2.8% of CVCs were associated with CVC-associated BSI
• Factors associated with CVC-BSI were: TPN with non-
impregnated CVC (95% CI 1.69-2.88, p < 0.0001), absent o
antibiotics for 48 hours after insertion of CVC (95% CI: 1.39
2.72, p = 0.0001), patient unarousable ≥ 70% (CI: 1.38-2.3
p < 0.0001).
• The proportion of float nurse-days > 60% found to be
independent risk factor (p = 0.0019)
• Risk for CVC-associated BSI was lower with antimicrobial-
impregnated CVCs
• The risk of CVC-BSI was not associated with N/P ratio or th
patient-care assistant-to- patient ratio
• The risk of CVC BSI was 2.6 times higher for CVCs inserte
patients cared for by float nurse more than 60% of time (7 o
884, 7.92 BSIs per 1000 CVC days, p = 0.01).

Whitman et al/ 2002, To describe the rates of
USA selected nurse sensitive
patient outcomes and
to determine if there
are differences in rates
across units
• Secondary • CLBSI: number of CLIs/ • Describes and • CL infection and PUs were higher in non-cardiac ICUs (p =
analyses of monthly number of central line compares the rates 0.037)
prospective days in place of already recognized • Falls rates were higher in MS units comparing with NCICUs
observational data • PU: number of hospital nurse sensitive patient and cardiac ICU (CICUs) (p = 0.035 and p = 0.003, respect
• 95 patient care units acquired PUs (grade outcomes by ANA • Satisfaction with pain had minimal variability ranging from 0
across 10 adult II or greater)/number of to 0.57
acute care hospitals patients assessed for • Restraint application duration was higher in NCICUs and C
• Data collection skin break down (p=0.001)
methods were • Medication errors: • Medication errors were ranging from 0.1 to 0.5
different for each number of reported • Significant differences were found with respect to mean CL
NSO (monthly medication errors/ rates (F[4.59] = 6.25, p = 0.001), pressure ulcer rates (F[4.8
surveillance data number of dispensed = 5.04, p = 0.001), fall rates (F[4.90] = 4.94, p = 0.001) and
from infection doses RADRs (F[4.75] = 12.6, p = 0.001)
control staff, monthly • Falls: number of • No significant differences were observed in medication erro
system wide 1 unplanned descents rates (F[4.59] = 6.25, p = 0.001) and satisfaction with pain
day prevalence, to the floor/ number of management by nurse rates (F[4.77] = 0.49, p = 0.7)
pharmacy and patient days
risk management • Patient satisfaction
reports, patient’ with pain management:
interviews one day percentage of patients
every month, finance responding very
office reports)/ satisfied
• 1 year study • Restraint application:
number of hours in
restraints/ number of
total hours available to
restrain patients

Table 1. Continued

31 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v

Authors, Research aim(s)
publication date,
country
Bracco et al., 2001, To determine the
Switzerland occurrence of critical
incidents focusing on
incidents due to human
factors. Secondary aim
was to identify patients or
situations at risk and try
to determine clinical and
financial consequences
of human related
incidents
Methods: study QIs and formula or Nurse staffing variables Results
design, population, definition identified and definition identified
tools used study
duration
• Prospective • Mortality • Human errors • Median LOS was 1.9 days, readmission rate was 4.3% and
observational study • Readmissions • Regular nursing staff hospital mortality was 5.9%: 1.4% among planned and 8.9%
• 1024 consecutive • Critical incidence amounted to 3.2 nurses among unplanned admissions (p < 0.0001, OR = 6.6, 95%
patients admitted • Cost per bed. 3.7-9.4). Predicted and observed mortality were found to be
in a 11- bed • LOS • Nursing staff included: 13.9% and 8.9% (p < 0.0001), respectively
multidisciplinary ICU • Human errors: mix skilled, ICU certified • 777 critical incidents were reported and 241 human errors
in a non- university • venous lines and nurses, certified nurses occurred in 161 patients
teaching 280- bed catheters, respiratory undergoing ICU training • Cost attributable to human error was estimated at 800,000
hospital system, cardiovascular and RNs without ICU euros per year and mean daily ICU cost per patient was 19
• A list including 105 system, drug related certification euros
items (defining complications, • ICU LOS was prolonged by 425 patient-days of treatment o
critical incidents) neurological system one year, due to errors (OR = 1.26, p = 0.0001)
and a standardized complications, urinary • Surveillance problems occurred after a median delay of
data sheet were system complications, 41 hours (p = 0.001) , planning errors had more severe
used to collect gastrointestinal consequences than execution or surveillance problems (p =
data during clinical system complications, 0.01)
round. Incidents skin and muscular • Risk of human error was correlated with severity of physiolo
were analyzed within system, management disturbance (p < 0.0001) and with LOS ICU (p < 0.0001)
24hrs complications • The overall risk of human error was 16% but in patients
• 1 year study • All definitions of already affected by human error, the risk of a second error
critical incidents increased to 30%
and categorization • ICP monitoring (40.0%, RR = 2.6, p = 0.05), LOS (OR =
of complications 1.26, p = 0.0001), readmission (OR = 3.04, p = 0.0005) and
were listed including Simplified Acute Physiology Score (OR = 1.22, p = 0.0034)
diagnostic criteria significantly associated with human error
• Patients were at higher risk of human error when an invasiv
technique was used: mechanical ventilation (26.4%, Relativ
risk [RR] = 2.8, p = 0.0001), pulmonary arterial catheter (35
RR = 2.7, p = 0.0001) and renal support (50.0%, RR = 3.4,
= 0.0001)

Valentin et al., 2006, To assess the prevalence
world-wide and corresponding
factors of selected
unintended events that
comprise patient safety
in ICU
• Observational, • Sentinel events. All
prospective, sentinel events were
multinational study presented as rates per
of incidents. Cross- 100 patient days
sectional design • Medication errors:
• 1913 patients adult - prescription;
> 18 years old in administration; wrong
205/280 ICUs from dose, drug, route
29 countries and 4 • Airway: unplanned
continents extubation - artificial
• Data were obtained airway obstruction; cuff
by: a structured leakage; prompting
questionnaire reintubation
was used for • Indwelling lines: iv
anonymous report of cannulas and the
unintended events; attachment fluid
a questionnaire delivery sets; catheters:
for information for arterial line, CVP,
ICU characteristics, pulmonary artery
patient and nurses’ catheters, folley,
related factors; and dialysis; probes and
the nine equivalents drains: unplanned
of nursing manpower dislodgment,
use score (NEMS) inappropriate,
for nursing workload; disconnection of chest
patient data which drain and nasogastric
were recorded using tubes
a project website and • Equipment failure:
online data collection infusion devices;
software ventilator and
• 24hr observation accessories; renal
period replacement devices;
power and oxygen
supply
• Alarms: inappropriate
turn off
• Nursing workload as
calculated by NEMS
• N/P ratio each shift
• Median P/N ratio ranged 1.3 to 2.0 and median P/physician
ranged from 3.0 to 6.0
• 584 sentinel events affected 391/1913 patients
• 38.8 events per 100 patient days were observed (95% CI
34.7-42.9)
• 268 patients experienced only one event, 123 patients > 1 e
and 1522 no event
• The most frequent events were related to lines, catheter an
drains occurred in 158 patients and the second most
frequent events were those associated with prescription an
administration of drugs
• 14.5 events related to lines, catheters and drains per 100
patient day (95% CI 12.0-16.9), 10.5 events related to
medication/100 patinet days (95% CI 8.6-12.4), 9.2 events
related to equipment/100 patient days (95% CI 7.4-11.1), 3
airway related events/100 patient days (95% CI 2.4-4.3) a
1.3 alarms related events/100 patinet days (95% CI 0.6-1.9
• Higher severity of illness, higher level of care, longer LOS in
ICU before observation and a longer duration of exposure w
associated with elevated ORs for experiencing a sentinel e
• There was an association of trauma ICUs with lower odds f
the occurrence of sentinel events (OR 0.47, 95% CI 0.22-1
p = 0.04)
• P/N ratio showed a slight nonlinear influence (p = 0.04,
quadratic term p = 0.006)

Tarnow-Mordi et al., To investigate whether
2000, Scotland mortality is independently
related to nursing
requirement and other
measures of workload
• Retrospective • Mortality: death in the • Measures of workload • 337 deaths were recorded (226 ICU, 111 before hospital
analysis from a ICU or before discharge in each patient’ stay per discharge) and 61 readmissions in a total of 1286 admissio
prospective cohort from hospital shift: occupancy per • Median LOS was 2.2 days (0.3-95.8)
study • LOS shift: the highest number • Median ratio of occupied beds to appropriately staffed beds
• 1050 adult episodes • Readmission rates in of ICU beds occupied was 1.3 (0.4-2.2)
representing the ICU each shift • Median nursing requirement per shift was 9.2 (2.5-14.9).
1025 patients (25 • Peak occupancy: the • Median nursing requirement per occupied bed per shift was
readmissions) in highest occupancy per 1.6 (0.7-2.3)
1 ICU shift during the patients’ • Adjusted mortality was related to the ratio of occupied of
• Locally agreed stay appropriately staffed beds per shift, peak occupancy and IC
formula was used to • ICU nursing requirement nursing requirement per occupied bed per shift
calculate the number per shift: the highest • Unadjusted mortality was greater for patients exposed to hi
of appropriated number of nurses moderate overall ICU workload (OR = 4.0, 95% CI 2.6-6.2)
staffed beds required for the • Adjusted mortality was more than 2 times higher in patients
• Nursing requirement ICU according the exposed to high workload (average nursing requirement pe
per shift was recommendations of UK occupied bed and peak occupancy) than those exposed to
recorded by senior Intensive care society workload (OR = 3.1, 95% CI 1.9-5.0)
nurse at the end of • Number of appropriately
each shift according staffed beds: total whole
to recommendations time-equivalent nurses-
of UK intensive care • Nurse workload: the
society ratio of occupied to
• 4 year study appropriately staffed
beds

Table 1. Continued

33 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v

Authors, Research aim(s) Methods: study QIs and formula or Nurse staffing variables Results
publication date, design, population, definition identified and definition identified
country tools used study
duration
Garcia & Fugulin, To analyze the time • Quantitative • NGT loss: definition • Nursing time (including • Mean number of nursing hours were 13.9 hours per patient
2012, Brazil; USA the nursing team uses descriptive of ANA, National nurses and nursing day in 2008 and 14,1hrs/pat/day in 2009
to see patients in ICU, correlational study Database of Nursing technicians) spent to • - Mean care hours for nurses and nursing technicians rema
as well as to check its in an adult ICU QIs assist each patient: the same in 2008 and 2009
correlation with quality • All patients admitted • CVC loss: number of mean number of nursing • The proportion of nurses’ care time was found to be 31%. T
care indicators during study period CVC losses/ number of staff members in each care time of nursing technicians was 69%
• Selected QIs of the patients with CVC per professional category • Nursing care time spent and the QI “incidence of accidenta
Institution’s group day x 100 X mean productivity of extubation” showed statistically significant correlation (r =
of QIs that were • Incidence of pressure each professional -0.454, p = 0.026)
already validated ulcers: definition of category X work journey • The mean incidence of accidental extubation was found to
and recommended in ANA, National of each professional be 0.73 (SD 0.57) and 0.46 (SD 0.58) for 2008 and 2009,
Brazilian literature Database of Nursing QI category ÷ mean daily respectively
• The data • Extubation incidence: number of patients • The incidence of accidental extubation decreased when the
were monthly definition of ANA, attended nursing care time increased (r = -0,454, p = 0.026)
collected from National Database of
the management Nursing QI
instruments:
“Worksheet to
calculate the mean
nursing care time
spent” (part of head
nurses’ monthly
reports calculated
electronically by
equation in excel);
“Worksheet to obtain
nursing quality
indicators”/
• 2 year study

Metnitz et al., 2008, To evaluate the • Prospective,
Austria relationship between multicenter cohort
patient volume and study
outcome (survival status • 83259 patients from
at ICU and hospital 40 ICUs in Austria
discharge status) in a • A questionnaire was
large cohort of critically used to examine
ill patients the structural
characteristics
of quality of the
included ICUs
• ASDI prospectively
collected data
• 7 year study
• Fatal outcome
• Observed to expected
mortality: number of
observed deaths per
group/ the number
of SAPS II predicted
deaths per group
• ICU LOS
• Hospital LOS
• Level of the provided • LOS was 3 days (median Q1-Q2, 2-7)
care (assessed by • Observed/ expected mortality ratio was 0.90 (95% CI 0.89-
TISS 28) • An increase in the number of patients/year/ICU bed (turnov
• P/N ratio: number of (OR = 0.967, 95% CI 0.956-0979) and an increase in the
patients assigned to number of patients treated in the same diagnostic category
one nurse = 0.995, 95% CI 0.993-0.996) reduced the risk of fatal outc
• The efficient use of • An increase in P/N ratio (OR = 1.082, 95% 1.019-1.149) an
nursing personnel was increase in the number of diagnostic categories (OR = 1.06
evaluated from the work 95% CI 1.044-1.086) were associated with worse outcomes
utilization ratio and was • A significant positive correlation between later admission a
calculated by specific survival was observed (OR per year 0.96, 95% CI 0.95-0.9
formula • When P/N ratio was increased by 1 and the nurse had to
• Occupancy rate: the care for an additional patient, the risk of dying at hospital w
percentage of occupied increasing by 8% and by 30% in univariate and multivariate
beds per day analysis, respectively

Stone et al., 2007, To examine effects of a • Observational study • CLBSI: according to • RN hours per patient • 30-day mortality rate was 22%
USA comprehensive set of • 51 adult ICUs in 31 definition and formula of day • Overall rates for infection were low: CLBSI: 0.95% (61/6385
working conditions on hospitals, 15846 NNIS protocols • Ratio of overtime to CAUTI: 1.7% (102/6031), VAP: 1.5% (81/5462). Average 3
elderly patient safety elderly ICU patients • VAP: according to regular time hrs of RNs day mortality was 22% and decubitus ulcer 2.0
outcomes in ICU • Standardized data definition and formula of • Average RN wages • Patients admitted to ICUs in which nurses perceived more
collection forms NNIS protocols per ICU positive organizational climate had slightly higher odds for
were used: data • UTI: according to • Average RN wages developing CLBSI (adjusted OR = 1.19, 95% CI 1.05-1.36)
were collected definition and formula of per ICU were 39% less likely to develop CAUTI (adjusted OR = 0.6
by Medicare files (30 NNIS protocols • Overtime 95% CI 0.44-0.83)
day mortality, • 30 day mortality: the • Effective work conditions • Patients admitted to ICU with more RN hours per patient
decubiti), NNIS (for date of index admission (by the organizational day had significantly lower incidence of CLBSI, VAP, 30-da
CLBSI, VAP, CAUTI), in inpatient standard climate calculated by mortality and decubiti (p ≤ = 0.05)
administrative data analytic file to the Nursing work scale • In settings where nurses worked less overtime patient
(covariates), AHA date of death in the (NWS) experienced less CLSBI (adjusted OR = 0.33, 95% CI 0.15
,and RN survey (for denominator file, 0.72)
the organizational • Decubiti: according to • In settings where nurses worked more overtime patients ha
climate by Nurse definition and formula of increased odds in CAUTI (p<0.001) and higher rates of dec
work environment AHRQ protocol (adjusted OR = 1.91, 95% CI 1.17-3.11)
scale- 42 item) • Hospitals with the lowest profit margin had less adverse
• For administrative outcomes (CAUTI, VAP, decubiti) than those more profitab
processes the ≤ 0.05), whereas CLBSI had negative relationship (p < 0.00
monthly payroll data • Increased overtime was associated with patients’ risk of CA
was used ( RN hours decubitus ulcer. Less overtime was associated with lower
per patient day, incidence of CLBSI
ratio of overtime to
regular time hours of
RN and average RN
wages per ICU
• Medicare cost
reports were used
to estimate profit
margin
• Magnet accreditation
status was
determined using
credentialing body’s
website
• During the year 2002

Table 1. Continued. AHA = American Hospital Association (annual survey data), ANA = American Nurses Association, ASDI = Austrian Center for Documentation and Quality Assurance, BSI =
bloodstream infection, CAUTI = catheter-associated urinary tract infection, CDC = Centers of Disease Control and Prevention, CI = confidence interval, CIMC = cardiac intermediate care units, CCLI
= central catheter line infection, CLBSI = central line bloodstream infections, ICD= International Center of Disease, ICU = i ntensive care unit, ICP = intracranial pressure , IMC = intermediate care unit,
LOS = length of stay, MAE= medication administrator error, MS = medical/surgical, NCICU = Non-cardiac ICU, NEMS = Nine Equivalents of Nursing Manpower Use Score, NNIS = National Nosocomial
Infections Surveillance, NNPR = night nurse to patient ratio, N/P ratio = nurse to patient ratio, NWS = Nursing Work Scale , NNIS = nosocomial infections surveillance, OR = odds ratio, p = significance
value, RADR = restraint application duration rate, RN = registered nurse, SAPS= Simplified Acute Physiology Score, UE = unplanned extubation, UTI = urinary tract infections, Yrs = Years, Vs = Versus

33 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Category οf outcome

outcome variables
Sub-category οf

Valentin et al., 2006

Metnitz et al., 2008

Bracco et al., 2001
Robert et al., 2000
Kendall-Gallagher

Alonso-Echanove

Garcia & Fugulin,

QIs identified

Stone et al., 2007

Dang et al., 2002

Tarnow-Mordi et
Amavardi et al.,
Cho et al., 2008
& Blegen, 2009

Whitman et al.,
variables

et al., 2003

al., 2000
Studies identified

2000

2002

2012
Mortality (adjusted and unadjusted,
X X X X X
Setting
observed and expected- SMR)/fatal
related
outcome
Primary QIs

Setting
30 day mortality
X
related

Patient
related use X X X X X X
of health ICU LOS
care

Patient
related use X X X
of health Hospital LOS
care

Setting
related Cost
X X

Patient
related use X X
of health
Readmission
Secondary QIs

care
Infections

UTI X X

CVC-associated BSI/ CLBSI X X X X

BSI X X
Patient
related Postoperative septicemia X X
safety
Postoperative infection X
Post-operative pneumonia X
Adverse events
Secondary QIs

VAP X
Medication errors X X X
Skin break down, pressure sores X X X X

Patient Falls X X
related Reintubation X X
safety
Extubation incidence/unplanned X X
extubation
Complication after OR

Aspiration X X
Secondary QIs

Acute MI X X
Cardiac arrest X X
Complications after procedure X
Pulmonary insufficiency after X
Patient
related procedure
safety Ventilation > 96 hours X
Renal failure X X
Complication after OR

Platelets transfusion X
Secondary QIs

Cardiac complications X

Reoperation for bleeding X

Surgical complications X

Patient
related
safety Critical incidence X

Table 2. Categorization of quality indicators identified in the included studies

35 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Category οf outcome

outcome variables
Sub-category οf

Valentin et al., 2006

Metnitz et al., 2008

Bracco et al., 2001
Robert et al., 2000
Kendall-Gallagher

Alonso-Echanove

Garcia & Fugulin,

QIs identified

Stone et al., 2007

Dang et al., 2002

Tarnow-Mordi et
Amavardi et al.,
Cho et al., 2008
& Blegen, 2009

Whitman et al.,
variables

et al., 2003

al., 2000
Studies identified

2000

2002

2012
Human errors related to venous lines X
and catheters

Human errors related to respiratory X

system
Secondary QIs

Human errors related to X

Errors

cardiovascular system

Human errors related to drug related X

complications

Patient Human errors related to neurological X

related system complications
safety
Human errors related to urinary X
system complications

Human errors related to X

Gastrointestinal system
complications

Human errors related to skin and X

muscular system
Secondary

Human errors related to X

QIs

management complications

Patient Patient satisfaction with pain X

related management
perception
Sentinel/unintended events

Patient Restraint application X

related
Secondary QIs

safety

Sentinel events related to medication X

(prescription, administration)

Sentinel events related to airway X

Patient Sentinel events related to indwelling X X

related lines (lines, catheters, drains, arterial
safety lines CVP lines, pulmonary artery
catheters, folley catheters, dialysis
catheters, chest drain, nasogastric
tubes)

Sentinel events related to equipment X

Table 2. Continued

DISCUSSION group in the ICU, as well as the diagnostic related group are factors
that may affect the sensitivity to nursing care and the selection of QIs.
The current review builds on previous research pooling patients’ QIs that describe the positive effect of nursing care delivered by
outcomes associated with the quality and quantity of nursing care measures of improved health status, such as symptom control (e.g.
in the ICU. This review examines the methodological quality of QIs dyspnea, nausea) were not identified. This finding may suggest
identified, through a relative new instrument (AIRE) (De Roo et al., that this type of information is not recorded and documented in
2013). To the authors’ knowledge, this is the first time that the AIRE administrative databases (Savitz et al., 2005). Only Whitman et al.
instrument is used for the methodological quality assessment of NQIs (2002) reported rates of “patient satisfaction with pain management”.
in the ICU. Likewise, there seems to be a lack of patient- reported outcome QIs,
The QIs identified concerned a specific health care setting (ICU), which should be addressed in future studies.
even though they were previously used for other health care settings Methodological characteristics of the identified QIs varied
as well. The heterogeneity was evident both with regards to the considerably across the included studies (Table 3). The high scores
number of QIs, which ranged from 1 to 15, as well as to the type of obtained in the category “scientific evidence” were indicative of the fact
QIs measured (Table 1 and 2). The observed variation across QIs that generally valid and widely used QIs were included in these
indicates that the selection of QIs depends to some extent on the type studies. On the other hand, low scores in the remaining categories
and needs of each ICU, as well as the purpose the QIs are selected suggest that future studies should pay more attention to reporting the
to serve. The composition of involved stakeholders and the defined involvement of relevant stakeholders as well as to providing full
criteria may also affect the selection of QIs. The geographical origin, descriptions of the QIs used.
the main causes of mortality of the population under study and the
Structure variables associated with patient outcomes differ among
available means need to be taken into consideration (Mainz, 2003).
included studies (Table 1). This was also highlighted in previously
Additionally, critically ill patients’ outcomes are not equally sensitive
to nursing care (Whitman et al., 2002). The examination of a specific published reviews (Numata et al., 2006, Penoyer, 2010, West et al.,

35 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v

Standardized category score

Item score (average ratio) %
Tarnow-Mordi et al., 2000

% (min: 0 , max: 100)

Alonso-Echanove et al.,

Garcia & Fugulin, 2012

(min: 25, max: 100)

(min: 25, max:100)
Amavardi et al., 2000

Category score %
Whitman et al., 2002
Kendall-Gallagher &

Valentin et al., 2006

Metnitz et al., 2008

Bracco et al., 2001
Robert et al., 2000

Stone et al., 2007

Dang et al., 2002
Cho et al., 2008
Blegen, 2009
Studies identified

2003
Items Dimension
Purpose, relevance and
organizational context

The purpose of the indicator is described

clearly
11 15 16 12 12 15 15 14 14 15 16 14 11 87

The criteria for selecting the topic of the

indicator are described in detail
10 11 13 11 14 14 15 13 13 14 14 13 12 80

The organizational context of the indicator is

described in detail 8 9 10 11 9 10 9 12 13 9 10 11 12 64 81 75
The quality domain the indicator
addresses is described in detail 13 6 16 16 15 15 12 15 15 15 12 11 14 83
Stake holder
involvement

The health care process covered by the

indicator is described and defined in detail 14 13 15 14 14 15 15 15 15 15 15 16 16 92
The group developing the indicator
includes individuals from all relevant 4 4 4 9 10 4 4 4 4 6 4 4 4 31
professional groups

Considering the purpose of the

indicator, all relevant stakeholders have
51 35
Scientific evidence

been involved at some stage of the 4 4 4 16 9 8 4 4 4 5 4 4 4 36

development process

The indicator has been formally endorsed 16 4 16 16 15 14 4 15 15 13 16 12 16 88

Systematic methods were used to search
for scientific evidence 11 11 14 12 13 15 13 13 13 14 14 13 16 83
The indicator is based on
recommendations from an evidence
based guideline or studies published in 16 16 16 11 16 16 15 15 15 16 16 15 16 96 89 85
peer-reviewed scientific journals
Additional evidence, formulation, usage

The supporting evidence has been

critically appraised 16 16 15 11 14 14 15 14 14 13 14 14 15 89
The numerator and denominator are
described in detail 6 15 16 10 10 15 14 10 10 14 13 13 12 75
The target patient population of the
indicator is defined clearly 11 15 16 15 16 16 11 12 12 13 12 14 16 86
A strategy for risk adjustment has been
considered and described 8 16 14 16 14 14 10 13 13 13 10 13 15 81
The indicator measures what it is
intended to measure (validity) 14 14 16 12 14 16 12 14 14 16 14 12 16 88
The indicator measures accurately and
consistently (reliability) 14 14 15 12 14 16 15 14 14 16 86 82
14 11 16 89
The indicator has sufficient discriminative
power 14 12 14 15 11 14 13 12 12 13 11 14 15 82
The indicator has been piloted in practice 16 16 16 16 16 16 16 14 14 16 14 15 16 97
The efforts needed for data collection
have been considered 14 15 15 16 15 15 16 14 14 14 14 14 15 92
Specific instructions for presenting and
interpreting results 11 16 15 12 14 15 14 14 14 14 13 13 15 87
Table 3. Quality assessment of the quality indicators identified through research and evaluation (AIRE) instrument (De Roo et al., 2013). Category score = (sum of individual authors’ scores for the items in a
category) / (maximum possible score for that category); Standardized category score (range 0-100%) = (total score per category - minimum possible score per category) / (maximum possible score per
category - minimum possible score per category)

2009). Almost half of the studies examined N/P ratio. Nevertheless, it observed (Table 1). This variation may be related to differences
should be noted that, even when the same nurse staffing variable was regarding organizational and structural factors of the ICUs, including
considered, it was measured in different ways, for example N/P ratio in available resources, capacity, type of each ICU, as well as care
the morning versus night shift (Amavardi et al., 2000, Dang et al., processes and policies, for instance admission and discharge criteria
2002). It is evident that there are many different ways of measuring (Pronovost et al., 2003). Furthermore, it is of note that there are
nurses’ contribution to patient care. However, this variability makes no universally applied systems that enable constant collection of
comparison among the studies difficult. It is of note that in more than QIs. Similarly, there are no uniform definitions and descriptions of
half of the studies, it is not clear if nurses are registered nurses (RNs) or QIs (Whitman et al., 2002). Thus, similar QIs may lead to different
if other levels of nursing personnel were included. calculations because the formulas as well as inclusion and exclusion
A wide variability regarding mortality and morbidity rates was also criteria provided by different organizations may vary (AHRQ 2006).

37 2018 Volume 12 Number 2 The World of Critical Care Nursing

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
The findings of this review suggest that the increased ratio of
average daily census to the total number of full time equivalent RNs
(ADC/RNs) and N/P ratio at day and night, overtime, workload, use of
float or pool nurses and the low intensity staffing are all strongly
associated with higher rates of adverse events, mortality, infections
and complications in adult ICUs (Table 1). These findings are
consistent with previous results in acute and critical care populations
(Griffiths et al., 2008, Numata et al., 2006, Penoyer, 2010, Van den
Heede et al., 2007, West et al., 2009).
Sets that assess ICU performance have been previously described
(Najjar-Pellet et al., 2008). Furthermore, there are sets of QIs that
focus on specific populations of the ICU: neurological (Russell et al.,
2002), end-stage (Clarke et al., 2003), palliative care (Mularski et al.,
2006), medical and surgical (Pronovost et al., 2003) and cardiology
patients (Idemoto and Kresevic, 2007). The current literature review
did not reveal a complete set of QIs according to structure-process-
outcome, which can quantify nursing care in the ICU. However, the
NQIs identified are evidence-based and valid. Thus, they could be
used in the context of the development of a unique set of NQIs
specific to the ICU. To achieve this end-goal, a well-defined multi-
level and structured research approach is needed that combines the
available evidence with expert opinion.
Limitations
The use of MESH terms probably limited retrieval of results
(Pronovost et al., 1999). In particular, using the general term QI as
a key word probably restricted the articles yielded. Furthermore,
the review did not employ any method for retrieving grey literature.
Even though a number of additional studies from European countries
(Isfort, 2013) and Korea (Choi et al., 2008) were identified in the
search, those were not published in the English language, and were
thus excluded. To the authors’ knowledge, the AIRE instrument has
not been used previously in the context of assessing QIs in the
ICU. Thus, no comparisons could be drawn. Lastly, the inclusion of
articles published after 2000 may have limited the number of articles
examined; however, the review has covered a relatively long time
period.
CONCLUSIONS
ICU patients may be at greater risk when the overall nursing staff
performance is disturbed (West et al., 2009). Mishaps indicate poor
quality of the care provided. Continuous assessment of measures,
which quantify the level of nursing quality and nursing contribution
to patients’ care outcomes is pivotal. The NQIs pooled in this review
could be used for the development of a unique set specific to the
ICU. Monitoring of validated NQIs may improve critically ill patients’
outcomes that are mostly affected by nurses. This would also allow the
comparison between ICU settings. A set of NQIs specific for the ICU
may influence clinical nursing practice, guide improvements of the
care delivered and contribute to the transformation and improvement
of the health care system (De Vos. et al., 2007, Numata et al., 2006,
Penoyer, 2010). Lastly, monitoring of NQIs in a unit recognizes that
the critically ill patient remains the focus of the care process.
REFERENCES
Agency of Health Care Research and Quality (2006). AHRQ Quality
indicators. Guide to patient safety indicators, Version 3. [Online] Available
at: http://www.qualityindicators.ahrq.gov Αccessed 5 May 2007.
Alonso-Echanove J, Edwards JR, Richards MJ, et al. (2003). Effect of nurse
staffing and antimicrobial-impregnated central venous catheters on the
risk for bloodstream infections in intensive care units. Infection Control
and Hospital Epidemiology 24(12), 916-925.
Amaravadi RK, Dimick JB, Pronovost PJ, Lipsett PA (2000). ICU nurse-
to-patient ratio is associated with complications and resource use after
esophagectomy. Intensive Care Medicine 26(12), 1857-1862.
37
American Nurses Association (2000). Nurse staffing and patient outcomes in
the inpatient hospital setting. American Nurses Publishing: Washington.
Bracco D, Favre JB, Bissonnette B, et al. (2001). Human errors in a
multidisciplinary intensive care unit: a 1-year prospective study. Intensive
Care Medicine 27 (1), 137-145.
Braun JP, Hendrik M, Hanswerner B et al. (quality network in intensive care
medicine) (2010). Quality indicators in intensive care medicine: why? Use
or burden for the intensivist. GMS German Medical Science 8(22), 1-20.
Brook RH, McGlynn EA, Cleary PD (1996). Quality of health care. part 2:
measuring quality of care. New England Journal of Medicine 335 (13),
966-70.
Center for Reviews and Dissemination (2009). Systematic reviews. CRD’s
guidance for undertaking reviews in health care. CRD, University of York:
York. pp. 6-90.
Cho SH, Hwang JH, Kim J (2008). Nurse staffing and patient mortality in
intensive care units. Nursing research 57(5), 322-330.
Choi YJ, Lim JY, Lee YW, Kim HS (2008). Development of nursing key
performance indicators for an intensive care unit by using a balanced
score card. Journal of Korean Academic Nursing 38(5), 656-66.
Clarke EB, Curtis JR, Luce JM, Levy M, Danis M, Nelson J, Solomon MZ
(2003). Quality indicators for end-of-life care in the intensive care unit.
Critical Care Medicine 31 (9), 2255-62.
Dang D, Johantgen ME, Pronovost PJ, Jenckes MW, Bass EB (2002).
Postoperative complications: does intensive care unit staff nursing make a
difference? Heart and Lung 31(3), 219-228.
De Bruin-Kooistra M, Amelink-Verburg MP, Buitendijk SE, Westert GP
(2012). Finding the right indicators for assessing quality midwifery care.
Intnational Journal of Quality in Health Care 24(3), 301-310.
De Roo ML, Leemans K, Claessen SJ, et al. (2013). Quality indicators for
palliative care: update of systematic review. Journal of Pain and Symptom
Management 46(4), 556-572.
De Vos M, Graafmans W, Keesman E, Westert G, Van der Voort PH (2007).
Quality measurement at intensive care units: which indicators should we
use? Journal of Critical Care 22(4), 267- 274.
Delgado MCM, Pericas LC, Moreno JR, et al. (2005). Quality indicators in
critically ill patients. Madrid: SEMICYUC.
Donabedian A (1992). The role of outcomes in quality assessment and
assurance. Quality Review Bulletin 18(11), 356-60.
Doran DM (2003). "Preface", In: Doran DM (editor), Nursing-Sensitive
Outcomes: State of the Science. MA: Jones and Bartlett, Sudbury, pp.
vii-ix.
Doran DM, Pringle D (2011). Patient outcomes as accountability, In: Doran
DM (editor), Nursing Outcomes: The State of the Science, 2nd edition.
MA: Jones and Bartlett Learning, Sudbury, pp. 11-20.
Garcia PC, Fugulin FMT (2012). Nursing care time and quality indicators for
adult intensive care: correlation analysis. Revista Latino-americana de
Enfermagem 20(4), 651-658.
Griffiths P, Jones S, Maben J, Murrells T (2008). State of the art metrics for
nursing: a rapid appraisal. National Nursing Research Unit. King’s College
London: University of London, pp. 2-16. [Online] Available at: http://www.
kcl.ac.uk/schools/nursing/nnru.
Heslop L, Lu S (2014). Nursing-sensitive indicators: a concept analysis.
Journal of Advanced Nursing 70(11), 2469-2482.
Holzemer WL (1994). The impact of nursing care in Latin America and the
Caribbean: a focus on outcomes. Journal of Advanced Nursing 20(1),
5-12.
Idemoto Β, Kresevic D (2007). Emerging Nurse-Sensitive Outcomes and
Evidence-Based Practice in Postoperative Cardiac Patients. Critical Care
Nursing Clinics of North America 19(4), 371-384.
Isfort M (2013). Influence of personnel staffing on patient care and nursing
in German ICUs. Descriptive study on aspects of patient safety and stress
indicators of nursing. [German]. Medizinischa Klinik, Intensivmedizin und
Notfallmedizin 108(1), 71-77.
Joint Commission (2007). Performance measurement initiatives. The
JointCommission. [Online] Available at: www.jointcommission.org/
PerformanceMeasurement/PerformanceMeasurement/ Accessed 27 May
2007.
2018 Volume 12 Number 2 The World of Critical Care Nursing
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v

39 2018 Volume 12 Number 2 The World of Critical Care Nursing

View publication stats

 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Kendall- th Care
Gallag 15(suppl. 1), 5-11.
her D, McGahan M, Kucharski G, Coyer F (2012). Nurse staffing levels and the incidence of mortality and
Blegen morbidity in the adult intensive care unit: a literature review. Australian Critical Care 25(2), 64-77.
MA Metnitz B, Metnitz PGH, Bauer P, Valentin A (2008). Patient volume affects outcome in critically ill
(2009). patients. Wiener Klinische Wochenschrift. The Middle European Journal of Medicine 121(1-2), 34-
Compe 40.
tence
and Mitchell PH (2008). Defining patient safety and quality care, In: Hughes, R.G. (ed), Patient safety and
certific quality: An evidence-based handbook for nurses. AHRQ (US): Rockville.
ation of Montalvo I (2007). The national database of nursing quality indicators (NDNQI). Home ANA
registe Periodicals, OJIN: the on online journal of issues in nursing 12(3).
red Mularski RA, Curtis JR, Billings JA, et al. (2006). Proposed quality measures for palliative care in the
nurses critically ill: A consensus from the Robert Wood Johnson Foundation. Critical Care Workgroup.
and Critical Care Medicine
safety 34(11), 404-411.
of
patient Najjar-Pellet J, Jonquet O, Jambou P, Fabry J (2008). Quality assessment
s in in intensive care units: proposal for a scoring system in terms of structure and process. Intensive
intensi Care Medicine 34(2), 278-85.
ve care National Quality Forum (2004). National Voluntary Consensus Standards for Nursing-Sensitive Care:
units. An Initial Performance measure set. A consensus report by the National Quality Forum. NQF,
Americ Washington: USA, pp. 1-14.
an
Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K (2002).
Journal
Nurse-staffing levels and the quality of care in hospitals. The New England
of
Journal of Medicine 346(22), 1715-1722.
Critical
Numata Y, Schulzer M, Van der Wal R., et al. (2006). Nurse staffing levels and hospital mortality in
Care critical care settings: literature review and meta- analysis. Journal of Advanced Nursing 55(4), 435-
18(2), 48.
106-
114. Penoyer DA (2010). Nurse staffing and patient outcomes in critical care: A
Mainz concise review. Critical Care Medicine 38(7), 1521-1528.
J Petrosyan Y, Sahakyan Y, Barnsley J, Kuluski K, Liu B, Wodchis W (2017). Quality indicators for care
(200 of osteoarthritis in primary care settings: a systematic literature review. Family Practice 35(2), 151-
3). 159.
Dev Pronovost PJ, Berenholtz SM, Ngo K, et al. (2003). Developing and pilot testing quality indicators in the
elopi intensive care unit. Journal of Critical Care
ng 18(3), 145-155.
evid
ence Pronovost PJ, Jenckes MW, Dorman T, et al. (1999). Organizational characteristics of intensive care
- units related to outcomes of abdominal aortic surgery. Journal of the American Medical Association
base 281(14), 1310-
d 1317.
clinic Ray B, Samaddar DP, Todi SK, Ramakrishnan N, John G, Ramasubban S (2009). Quality indicators for
al ICU: ISCCM guidelines for ICUs in India. Indian Journal of Critical Care Medicine 13(4), 173-206.
indic [Online] Available at: http://www.ijccm.org/text.asp?2009/13/4/173/60172
ators
:a
state
of
the
art
meth
ods
prim
er.
Inter
natio
nal
Jour
nal
for
Qual
ity in
Heal
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Robert J, Bernard S (2005). Quality indicators sensitive
Fridkin SK, to nurse staffing in acute care settings, In: Henriksen K, Battles JB, Marks ES, Lewin DI (Eds),
Blumberg Advances in patient safety: from research to implementation, Volume 4: programs, tools and
HM, et al. products. Rockville: Agency for Healthcare Research and Quality (US), pp. 375-385.
(2000). The Smeulers M, Verweij L, Maaskant JM, et al. (2015) Quality indicators for safe medication preparation
influence of and administration: a systematic review. PLOS One 10(4), 1-14. [Online] Available at:
the https://doi.org/10.1371/journal. pone.0122695.
compositio
Stone PW, Mooney-Kane C, Larson EL, et al. (2007). Nurse working conditions and patient safety
n of the
nursing outcomes. Medical Care 45(6), 571-578.
staff on Strudwick K, Nelson M, Martin-Khan M, Bourke M, A Bell, Russell T (2015). Quality indicators for
primary musculoskeletal injury management in
bloodstrea the emergency department: a systematic review. Academic Emergency
m Medicine 22(2), 127-141.
infection
rates in Tarnow-Mordi WO, Hau C, Warden A, Shearer J (2000). Hospital mortality
a surgical in relation to staff workload: a 4 year study in an adult ICU. The Lancet
intensive 356(9225), 185-189.
care unit. Valentin A, Capuzzo M, Guidet B, et al. (2006). Patient safety in intensive care: results from the
Infection multinational sentinel events evaluation (SEE) study. Intensive Care Medicine 32(10), 1591- 1598.
Control
Van den Heede K, Clarke SP, Sermeus W, Vleugels A, Aiken LH (2007). International experts’
and
perspectives on the state of the nurse staffing and patient outcomes literature. Journal of nursing
Hospital
scholarship 39(4), 290-297.
Epidemiol
ogy West E, Mays N, Rafferty AM, Rowan K, Sanderson C (2009). Nursing resources and patient
21(1), 12- outcomes in intensive care: a systematic review of the literature. International Journal of Nursing
7. Studies 46(7), 993-1011.
Russell West S, King V, Carey TS, et al. (2002). Systems to rate the Strength of scientific evidence, in:
D, Prepared by the Research Triangle Institute- University of North Carolina Evidence-based Practice
Vorder Center under Contract No. 290-97-0011. Evidence Report/Technology Assessment No.
- 47 AHRQ. Publication No. 02-E016. Rockville: Agency for Healthcare
Brueg Research and Quality, pp. 17-68.
ge M,
Whittemore R, Knafl K (2005). The integrative review: updated methodology.
Burns Journal of Advanced Nursing 52(5), 546-553.
S
(2002). Whitman GB, Kim Y, Davidson LJ, Wolf GA, Wang SL (2002). Measuring nurse- sensitive patient
Effect outcomes across specialty units. Outcomes Management 6(4), 152-158.
of an
outco
mes
manag
ed
approa
ch to
care of
neuros
n
cience
patient
s by
acute
care
nurse
practiti
oners.
Americ
an
Journa
l of
Critical
Care
11(4),
353-
362.
Savitz LA,
Jones CB,
 vIdentifying validated nursing quality indicators for the intensive care unit: an
integrative review v
Downloaded by Doctor Dawn Connolly At 07:34 18 September 2017 (PT)

39 2018 Volume 12 Number 2 The World of Critical Care Nursing

View publication stats