Академический Документы
Профессиональный Документы
Культура Документы
doi: 10.1093/intqhc/mzz005
Article
Address reprint requests to: Dr Yuanli Liu. Tel: +86-10-65105537; Fax: +86-10-65105537; E-mail: liuyl_fpo@126.com
Editorial Decision 22 December 2018; Accepted 16 January 2019
Abstract
Objectives: To develop a medical record-based, comprehensive system of healthcare quality indi-
cators for psychiatric hospitals in China.
Design: A modified Delphi process with analytic hierarchy process (AHP) was used.
Participants: Twenty nationally-recognized experts were invited to participate in two rounds of
Delphi expert consultation and AHP.
Methods: Fifty potential indicators were included based on literature review, and 20 experts were
asked to rate the importance of each indicator using two rounds of email surveys. The AHP was
used to determine the relative importance of the finalized quality indicators.
Results: The average authoritative coefficient was 0.92 ± 0.07. After two rounds of Delphi consult-
ation, 47 healthcare quality indicators were identified for Chinese psychiatric hospitals. The mean
importance ratings ranged from 7.06 to 8.84 on a nine-point scale, with variation coefficients ran-
ging from 0.04 to 0.22. The percentage of full score for potential indicators ranged from 16% to
74%. In two rounds, the Kendall’s W coefficients ranged from 0.423 to 0.535. The weights of struc-
ture, process and outcome were 0.175, 0.211 and 0.614, respectively.
Conclusion: We developed the first set of healthcare quality indicators for psychiatric hospitals in
mainland China, and it will provide a standardized and meaningful guide to evaluate the health-
care quality of psychiatric hospitals across the country.
Key words: healthcare quality, quality indicator, Delphi, analytic hierarchy process, psychiatric hospital, China
© The Author(s) 2019. Published by Oxford University Press in association with the International Society for Quality in Health Care. All rights reserved.
For permissions, please e-mail: journals.permissions@oup.com 1
2 Jiang et al.
However, measuring the healthcare quality of psychiatric hospitals International Accreditation Standards for Hospitals 6th edition [19]
is a major challenge, primarily due to a lack of accepted measure- and the Chinese Psychiatric Hospitals Accreditation Standards [17].
ment tools and feasibly accessible data. Second, before we began the modified Delphi process, we held a
For the past few decades, there have been major efforts to stand- consultation meeting with experts to refine the potential quality indica-
ardize the assessment of hospital performance, including psychiatric tors pool, based on data accessibility and feasibility. Six interdisciplin-
Developing medical record-based, healthcare quality indicators for psychiatric hospitals in China: a modified Delphi-AHP study
Indicator Round 1 Round 2
M ± SD Variation Percentage of Outcome Revised indicator M ± SD Variation Percentage of Outcome Revised indicator
full score (%) full score (%)
1. Structure quality 8.17 ± 0.58 0.07 45 Accepted 8.24 ± 0.93 0.11 42 Accepted
1.1 Hospital capability 8.39 ± 0.83 0.10 55 Accepted 8.47 ± 0.50 0.06 47 Accepted
1.1.1 Annual outpatient visits 7.89 ± 1.41 0.18 50 Accepted 7.53 ± 1.50 0.20 32 Accepted
1.1.2 Annual emergency visits 7.39 ± 1.74 0.24 45 Accepted 7.53 ± 1.55 0.21 32 Accepted
1.1.3 Annual emergency observing patients 6.94 ± 1.81 0.26 30 Deleted
1.1.4 Annual discharged patients 7.61 ± 1.53 0.20 40 Accepted 7.24 ± 1.49 0.21 21 Accepted
1.1.5 Open beds 7.44 ± 1.46 0.20 35 Accepted 7.35 ± 1.56 0.21 26 Accepted
1.1.6 Medical building office area 7.72 ± 1.41 0.18 45 Accepted 7.18 ± 1.35 0.19 21 Accepted
1.1.7 Staff number 7.17 ± 1.34 0.19 15 Accepted 7.82 ± 1.41 0.18 37 Accepted
1.1.8 Medical staff number 7.07 ± 1.90 0.27 16 Revised Proportion of 7.53 ± 1.54 0.20 37 Accepted
medical staff
1.1.9 Senior title medical staff number 7.10 ± 1.98 0.28 15 Revised Proportion of 8.00 ± 1.44 0.18 42 Accepted
senior title staff
1.2 Organization and IT system 8.28 ± 0.87 0.11 50 Accepted 8.24 ± 0.93 0.11 47 Accepted
1.2.1 AE reporting system 8.50 ± 0.60 0.07 55 Accepted 8.47 ± 0.65 0.08 53 Accepted
1.2.2 Medical quality management system 8.50 ± 0.60 0.07 55 Accepted 8.29 ± 0.74 0.09 42 Accepted
1.2.3 Independent medical quality management 8.39 ± 0.59 0.07 45 Accepted 8.41 ± 0.76 0.09 47 Accepted
department
1.2.4 Patient identity recognition system 8.50 ± 0.83 0.10 65 Accepted 8.53 ± 0.47 0.05 58 Accepted
2.Process quality 8.33 ± 1.00 0.12 55 Accepted 8.47 ± 0.63 0.07 53 Accepted
2.1 Disease assessment 8.78 ± 0.53 0.06 85 Accepted 8.71 ± 0.36 0.04 68 Accepted
2.1.1 Routine physical disease assessment 8.78 ± 0.42 0.05 80 Accepted 8.82 ± 0.27 0.03 74 Accepted
2.1.2 Routine psychiatric assessment 8.61 ± 0.76 0.09 70 Accepted 8.65 ± 0.83 0.10 68 Accepted
2.1.3 Routine social function assessment 8.61 ± 0.59 0.07 65 Accepted 8.53 ± 0.50 0.06 53 Accepted
2.1.4 Routine personality assessment 8.22 ± 0.92 0.11 50 Accepted 8.29 ± 0.74 0.09 42 Accepted
2.1.5 Routine suicide risk assessment 8.78 ± 0.42 0.05 80 Accepted 8.82 ± 0.36 0.04 74 Accepted
2.2 Medical process 8.17 ± 1.26 0.15 60 Accepted 8.41 ± 0.85 0.10 53 Accepted
2.2.1 Average number of types of medications 7.78 ± 1.96 0.25 50 Accepted 8.12 ± 1.08 0.13 42 Accepted
taken
2.2.2 Routine three level ward-round 8.50 ± 0.50 0.06 50 Accepted 8.59 ± 0.50 0.06 53 Accepted
2.2.3 Average psychotherapy times 8.06 ± 1.13 0.14 50 Accepted 7.94 ± 1.10 0.14 37 Accepted
2.3 Side effect monitoring 8.67 ± 0.58 0.07 70 Accepted 8.65 ± 0.61 0.07 63 Accepted
2.3.1 Regular monitoring of height and weight 7.44 ± 1.01 0.14 15 Accepted 7.47 ± 1.40 0.19 26 Accepted
2.3.2 Regular monitoring of blood glucose 8.00 ± 0.88 0.11 35 Accepted 8.18 ± 0.83 0.10 37 Accepted
2.3.3 Regular monitoring of blood lipid 7.89 ± 0.99 0.13 35 Accepted 8.12 ± 0.97 0.12 37 Accepted
2.3.4 Regular ECG monitoring 8.22 ± 0.85 0.10 45 Accepted 8.41 ± 0.51 0.06 42 Accepted
2.3.5 Regular monitoring of liver function 8.44 ± 0.83 0.10 60 Accepted 8.65 ± 0.47 0.05 58 Accepted
2.4 Medical quality training 8.67 ± 0.47 0.05 65 Accepted 8.71 ± 0.47 0.05 63 Accepted
2.4.1 Regular medical quality training for medical 8.61 ± 0.68 0.08 70 Accepted 8.53 ± 0.74 0.09 63 Accepted
staff
3.Outcome quality 8.78 ± 0.42 0.05 80 Accepted 8.84 ± 0.36 0.04 74 Accepted
3.1 Treatment effect 8.17 ± 0.96 0.12 45 Accepted 8.53 ± 0.50 0.06 53 Accepted
3.1.1 Average treatment response index in CGI 8.39 ± 0.76 0.09 55 Accepted 8.41 ± 1.09 0.13 53 Accepted
Table continued
3
4 Jiang et al.
Delphi Round 2
Average hospitalization
The results of the first round and the comments received were com-
Revised indicator
piled. Based on comments from the first round, the second round
indicators were developed. The same 20 experts were invited to par-
drug cost
ticipate in the second round to complete the questionnaires.
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
After two rounds of Delphi consultation, the quality indicators were
Revised
finalized. The same Delphi consultation experts were invited to con-
firm the finalized results and to complete a comparison matrix. At
each level, we set pairs for each indicator to others in the same level
full score (%)
Percentage of
[25]. For each pair of indicators, experts indicated which one was
16
21
21
37
37
32
42
63
21
21
more important and to what extent, using a 1–9 ordinal scale. The
extent for one indicator over another could be ‘absolute’ (9 points),
‘very strong’ (7), ‘strong’ (5), ‘moderate’ (3) or ‘equal’ (1). Intermediate
values are possible (2, 4, 6, 8) [24]. For example, if the extent of out-
Variation
0.21
0.22
0.20
0.11
0.20
0.14
0.10
0.07
0.11
0.14
come over structure was absolute, then the entry (outcome, structure)
of the matrix was 9, and the entry (structure, outcome) was 1/9.
7.06 ± 1.48
7.11 ± 1.56
7.08 ± 1.42
8.06 ± 0.86
7.65 ± 1.55
7.88 ± 1.10
8.29 ± 0.83
8.65 ± 0.61
7.59 ± 0.83
7.59 ± 1.07
Determining consensus
Round 2
M ± SD
Included in 3.1.1
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
Accepted
All data were entered into the IBM Statistical Package of Social
Deleted
Deleted
averaging the score of judgment basis (Ca) and the score of their
15
20
15
15
40
45
40
45
25
20
15
15
acquaintance with the questions (Cs) [23]. Table 2 shows the scor-
ing system which rated the basis of experts’ judgments. The Cs ran-
ged from 1.0 (very familiar) to 0.2 (unfamiliar).
Concentration and variation in Delphi experts’ opinions were
Variation
0.26
0.22
0.20
0.18
0.16
0.15
0.13
0.11
0.11
0.17
0.26
0.14
7.72 ± 1.24
7.83 ± 1.17
7.89 ± 1.05
8.17 ± 0.90
7.67 ± 0.88
7.33 ± 1.25
7.22 ± 1.88
7.50 ± 1.07
M ± SD
ant. The consistency ratio (CR) was also conducted. Low CR means
3.3.2 Average satisfaction for doctor-patient
3.2.2 Average hospitalization treatment cost
3.2.1 Average total hospitalization expenses
done again or the questionnaire had to be revised [25]. For this par-
3.3.1 Average global satisfaction
Results
3.4 Medical efficiency
Table 1 Continued
communication
Table 3 Characteristics of experts (N = 20) Table 4 The degree of coordination of expert opinions
Characteristic n % Kendall’s W χ2 P
Indicator Weight
mental health [29]. The quality indicators in several other countries Third, we selected a new outcome indicator, the treatment EI,
also include the population-based resources to measure the whole which is a component of the CGI scale, one of the most widely used
mental health service system [30]. On the other hand, some are assessments in clinical psychiatry. EI ranges from 1 to 16, according
rather narrow and limited, for example, we only found two sets of to treatment effect and side effect [39]. EI balances treatment effects
mental health indicators for forensic mental health hospitals in and adverse effects, making it a more ideal indicator to measure the
Japan and the Netherlands [31, 32]. Our indicators may be of inter- outcome of psychiatric treatments.
est to those who are interested in measuring the quality or perform- Fourth, this system is based on medical records rather than an
ance of psychiatric hospitals. IT system. Many of the mental health indicator systems in devel-
Second, different from other mental health indicator systems oped countries are based on IT systems or electronic medical
[27, 33–38], our system focused mostly on outcome quality. The records [40–42]. As China is vast and unevenly developed, the
outcome quality indicators had the highest weight (0.614). This majority of psychiatric hospitals in China have not adopted digital
indicates that the experts considered treating illness to be the most medical records. Therefore, instead of utilizing all patients’ elec-
important part in the medical quality monitoring process. tronic medical records, this indicator system is designed to be based
Developing medical record-based, healthcare quality indicators for psychiatric hospitals in China: a modified Delphi-AHP study 7
26. Tang X, Chen X, Pang Y et al. The development of quality indicators for 36. O’Brien AP, Boddy JM, Hardy DJ et al. Clinical indicators as measures of
home care in China. Int J Qual Health Care 2018;30:208–18. mental health nursing standards of practice in New Zealand. Int J Ment
27. Han H, Ahn DH, Song J et al. Development of mental health indicators Health Nurs 2004;13:78–88.
in Korea. Psychiatry Investig 2012;9:311–8. 37. Rossi G, Agnetti G, Bosio R et al. Italian quality assurance in mental
28. Essock SM, Olfson M, Hogan MF. Current practices for measuring men- health. J Nerv Ment Dis 2014;202:469–72.