John D. Merlino, Derek A. Fyfe, Angel R. Len and John N.

Oshinski
Brandon K. Fornwalt, William W. Sprague, Patrick BeDell, Jonathan D. Suever, Bart Gerritse,
Resynchronization Therapy
Agreement Is Poor Among Current Criteria Used to Define Response to Cardiac
Print ISSN: 0009-7322. Online ISSN: 1524-4539
Copyright 2010 American Heart Association, Inc. All rights reserved.
is published by the American Heart Association, 7272 Greenville Avenue, Dallas, TX 75231 Circulation
doi: 10.1161/CIRCULATIONAHA.109.910778
2010;121:1985-1991; originally published online April 26, 2010; Circulation.
http://circ.ahajournals.org/content/121/18/1985
World Wide Web at:
The online version of this article, along with updated information and services, is located on the

http://circ.ahajournals.org//subscriptions/
is online at: Circulation Information about subscribing to Subscriptions:

http://www.lww.com/reprints
Information about reprints can be found online at: Reprints:

document. Permissions and Rights Question and Answer this process is available in the
click Request Permissions in the middle column of the Web page under Services. Further information about
Office. Once the online version of the published article for which permission is being requested is located,
can be obtained via RightsLink, a service of the Copyright Clearance Center, not the Editorial Circulation in
Requests for permissions to reproduce figures, tables, or portions of articles originally published Permissions:
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
Arrhythmia/Electrophysiology
Agreement Is Poor Among Current Criteria Used to Define
Response to Cardiac Resynchronization Therapy
Brandon K. Fornwalt, PhD; William W. Sprague, BS; Patrick BeDell, BBA, RDCS;
Jonathan D. Suever, BS; Bart Gerritse, PhD; John D. Merlino, MD; Derek A. Fyfe, MD, PhD;
Angel R. Leon, MD; John N. Oshinski, PhD
BackgroundNumerous criteria believed to define a positive response to cardiac resynchronization therapy have been
used in the literature. No study has investigated agreement among these response criteria. We hypothesized that the
agreement among the various response criteria would be poor.
Methods and ResultsA literature search was conducted with the keywords cardiac resynchronization and
response. The 50 publications with the most citations were reviewed. After the exclusion of editorials and
reviews, 17 different primary response criteria were identified from 26 relevant articles. The agreement among 15
of these 17 response criteria was assessed in 426 patients from the Predictors of Response to Cardiac
Resynchronization Therapy (PROSPECT) study with Cohens -coefficient (2 response criteria were not
calculable from PROSPECT data). The overall response rate ranged from 32% to 91% for the 15 response criteria.
Ninety-nine percent of patients showed a positive response according to at least 1 of the 15 criteria, whereas 94%
were classified as a nonresponder by at least 1 criterion. -Values were calculated for all 105 possible comparisons
among the 15 response criteria and classified into standard ranges: Poor agreement (0.4), moderate agreement
(0.40.75), and strong agreement (0.75). Seventy-five percent of the comparisons showed poor agreement,
21% showed moderate agreement, and only 4% showed strong agreement.
ConclusionsThe 26 most-cited publications on predicting response to cardiac resynchronization therapy define response
using 17 different criteria. Agreement between different methods to define response to cardiac resynchronization therapy
is poor 75% of the time and strong only 4% of the time, which severely limits the ability to generalize results over
multiple studies. (Circulation. 2010;121:1985-1991.)
Key Words: heart failure

cardiac resynchronization therapy

pacing

pacemakers
P
redicting whether a patient will benefit, or respond, to
cardiac resynchronization therapy (CRT) has been the focus
of more than 500 publications during the last 5 years; however,
the definition of response to CRT varies widely between studies,
and numerous criteria to define a positive response to CRT exist
in the literature. Echocardiographic response is typically
assessed by quantifying the change in left ventricular ejection
fraction
14
or left ventricular end-systolic volume (LVESV)
2,510
3 to 6 months after CRT implantation. Clinical response is
assessed with the increase in the distance walked in 6 minutes
11
or improvement in New York Heart Association functional
class
2,1214
3 to 6 months after CRT implantation. Some studies
have defined response to CRT as a combination of several
clinical measures
1517
or as a combination of both clinical and
echocardiographic measures.
18
Editorial see p 1977
Clinical Perspective on p 1991
The heterogeneous approach to defining response to CRT
is a potential barrier to progress in this field. No study has
addressed this issue by investigating the agreement among
the numerous published CRT response criteria. If these
different response criteria show poor agreement, then the
ability to generalize results from multiple studies is severely
impaired, and a standard needs to be developed. We hypoth-
esized that the agreement between the various published
response criteria would be poor. We tested this hypothesis by
identifying response criteria from a literature search and then
assessing the statistical agreement among the different crite-
ria in the 426 patients enrolled in the Predictors of Response
to Cardiac Resynchronization Therapy (PROSPECT) study.
Continuing medical education (CME) credit is available for this article. Go to http://cme.ahajournals.org to take the quiz.
Received September 19, 2009; accepted February 17, 2010.
From the Emory University School of Medicine (B.K.F., W.W.S., J.D.M., D.A.F., A.R.L., J.N.O.), Atlanta, Ga; Emory/Georgia Institute of
Technology, Department of Biomedical Engineering (B.K.F., J.D.S., J.N.O.), Atlanta, Ga; The Carlyle Fraser Heart Center (P.B., J.D.M., A.R.L.),
Division of Cardiology, Atlanta, Ga; Medtronic Bakken Research Center (B.G.), Maastricht, the Netherlands; and Sibley Heart Center Cardiology
(D.A.F.), Atlanta, Ga.
Correspondence to Brandon Fornwalt, Emory University School of Medicine, Department of Radiology, MR Research, 1365 Clifton Rd, Atlanta, GA
30322. E-mail bkf@gatech.edu
2010 American Heart Association, Inc.
Circulation is available at http://circ.ahajournals.org DOI: 10.1161/CIRCULATIONAHA.109.910778
1985 by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
Methods
Literature Review
To identify commonly used criteria to define response to CRT, a
literature search was conducted with the Web of Science Science
Citation Index Expanded database
19
using the topics cardiac
resynchronization and response. The 50 publications with the
most citations were reviewed for relevance (Figure 1). Four review
articles and 20 publications that did not report individual response
criteria were excluded.
Seventeen different primary response criteria were identified from
the 26 remaining publications (Table 1).
118,2027
Eight of these 17
response criteria were based on echocardiography, 8 were based on
clinical measures, and 1 criterion was based on a combination of
both echocardiographic and clinical measures. Six of the 17 response
criteria included either all-cause or heart failure mortality as a
criterion to define a nonresponse, whereas the other 11 did not.
Patient Population
Agreement between response criteria was assessed with information
from the baseline and 6-month follow-up visits for the 426 patients
in the PROSPECT study.
10
Briefly, PROSPECT was a prospective,
multicenter study that was designed to test the ability of 12 different
echocardiographic dyssynchrony parameters to predict response to
CRT. Four hundred fifty-seven patients with standard CRT indica-
tions (New York Heart Association class III/IV heart failure, left
ventricular ejection fraction 35%, QRS 130 ms, and stable
medical regimen) were enrolled in PROSPECT at 53 centers
worldwide. After the exclusion of 31 patients who exited the study
early and did not receive an implant, 426 patients remained and were
followed up for 6 months after CRT implantation.
Statistics and Quantification of Agreement
The Cohen -coefficient was used to assess agreement between the
different response criteria. The -coefficient is an accepted statistical
coefficient that is used to assess agreement between methodolo-
gies.
28,29
The -coefficient ranges from1 (perfect disagreement) to
1 (perfect agreement), and a -coefficient of 0 indicates that the
amount of agreement was exactly that expected by chance.
28
A
-coefficient 0.75 was defined as strong agreement, 0.40.75
was defined as moderate agreement, and 0.4 was defined as poor
agreement (Table 2).
29
The -coefficient was calculated with the following equation:
(1)
observed agreement (%)expected agreement (%)
100%expected agreement (%)
Expected agreement is the percentage of cases in which 2 response
criteria would agree on the basis of chance alone and can be
calculated as follows:
(2) expected agreement (%)responders
criterion 1
responders
criterion 2
non-responders
criterion 1
non-responders
criterion 2
An example of a -value calculation is shown in Table 3 and
presented in the Results section.
Two of the 17 response criteria
24,25
could not be calculated from
PROSPECT data because (1) oxygen consumption at peak exercise
was not measured in PROSPECT and (2) height and weight at 6
months were not measured in PROSPECT, which precludes the
Figure 1. Flow chart showing the process by which response
criteria were identied from the literature.
Table 1. Seventeen Different Response Criteria Identified
From the 26 Relevant Publications
Response criteria
Echocardiographic
1. 1LVEF 5% (absolute)
1,2
2. 1LVEF 15%
3,4
3. 2LVESV 10% and did not die of progressive HF within 6
months
20,27
4. 2LVESV 15%
2,510
5. LVESV 115% of baseline
26
6. 2LVESVI 15%
25
7. 2LVEDV 15%
2
8. 1Stroke volume 15%
4,21,22
Clinical
9. 2NYHA 1
2,1214
10. 2NYHA 1 and did not die of progressive HF within 6 months
23
11. 2NYHA 1 and 16MWD 25%
15
12. 2NYHA 1 and 16MWD 25% and did not die of progressive HF
within 6 months
16,17
13. 16MWD 10%, no heart transplant, did not die of progressive HF
within 6 months
11
14. (2NYHA 1 or 1V

O
2
max 10% or 16MWD 10%) and alive, no
hospitalization for decompensated HF
24
15. Two of 3:
5
2NYHA 1
16MWD 50 m
2QOL 15
16. Clinical composite score improved
10
Combined
17. (1LVEF 5% absolute or 16MWD 30 m) and (2NYHA 1 or
2QOL 10)
18
1Indicates increase; LVEF, left ventricular ejection fraction; 2, decrease;
HF, heart failure; LVESV, left ventricular end-systolic volume; LVESVI, LVESV
indexed by body surface area; LVEDV, left ventricular end-diastolic volume;
NYHA, New York Heart Association functional class; 6MWD, 6-minute walk
distance; V

O
2
max, oxygen consumption at peak exercise; and QOL, quality-of-
life score.
If the authors did not specify whether death was considered a nonresponse,
then it was assumed that deaths were excluded.
1986 Circulation May 11, 2010
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
ability to calculate the LVESV index. Agreement among the 15
remaining response criteria was assessed by calculating a -value for
all possible pairs of criteria. Thus, 105 -values were calculated.
Group -values were then quantified (mean, median, and range) to
summarize the agreement among criteria from 4 different groups: (1)
All 105 comparisons, (2) comparisons between 2 echocardiographic
criteria, (3) comparisons between 2 clinical criteria, and (4) compar-
isons between an echocardiographic criterion and a clinical criterion.
Mean group -values were compared with a permutation test, and a
Bonferroni correction was applied to account for multiple compari-
sons. To justify that the sample size to estimate -values was large
enough, a bootstrap resampling procedure was used with a
Kolmogorov-Smirnov test to assess the normality of the resulting
distribution. P0.05 was defined as statistically significant.
Subgroup Analysis Excluding Response Criteria
Quantified Short Term and at 3 Months
One response criterion (increase in stroke volume 15%
4,21,22
) was
used in the literature as a short-term response measure that was
quantified within 2 days of CRT implantation. In addition, 2
response criteria from the literature were only used in studies that
assessed response at 3 months (criteria 15 and 17 in Table 1).
Short-term (2 days) and 3-month data for calculating all 15
response measures were not collected in PROSPECT, so all criteria
were assessed with data from the 6-month visit. To ensure that the
present results were not confounded by calculating these short-term
and 3-month measures from 6-month follow-up data, we performed
a subgroup analysis in which we excluded them and recalculated the
group -values.
Results
Response Rates
The percentage of patients defined as having a positive re-
sponse to CRT ranged from 32% to 91% for the 15 response
criteria (Table 4). All 15 criteria could be calculated in 250 of the
426 patients in PROSPECT. Of these 250 patients, 99% showed
a positive response according to at least 1 of the 15 criteria,
whereas 94% were classified as a nonresponder by at least 1
criterion. Similarly, 95% of patients showed a positive response
by at least 2 of the 15 criteria, whereas 87% were classified as
nonresponders by at least 2 criteria.
Example Calculation of -Value
An example of the -value calculation between response crite-
rion 3 (decrease in LVESV10%, no death due to heart failure)
and response criterion 13 (increase in 6-minute walk distance
10%, no heart failure death, no transplant) is given in Table 3.
Response criterion 3 identified 62% of patients who received
CRT as responders (and thus, 38% of the patients as nonre-
sponders), and criterion 13 also identified 62% of patients as
responders. The expected agreement due to chance alone was
therefore 0.620.620.380.3853%. The observed agree-
ment was 0.390.1554%. Equation 1 then shows that 0.02
for Table 3, which suggests poor agreement after accounting for
the level of agreement expected due to chance.
Agreement Among the 15 Response Criteria
The 15 response criteria showed poor agreement as a group
(Figures 2 and 3; mean 0.220.24, median0.14,
range0.2 to 0.97). Seventy-nine (75%) of the 105
-values were classified as having poor agreement, whereas
22 -values (21%) were classified as having moderate agree-
ment (Figure 3). Only 4 pairs of response criteria of the 105
total pairs (4%) showed strong agreement, and 2 of these 4
pairs were comparisons between a response criterion that
excluded mortality and the same exact criterion that defined
death as a nonresponse. The 7 echocardiographic response
criteria also showed poor agreement among each other
(Figure 4; mean 0.350.28, median0.29, range0.2
to 0.88). The 7 clinical response criteria showed moderate
agreement (Figure 4; mean 0.440.23, median0.43,
Table 2. -Values Used to Define Strong, Moderate, and
Poor Agreement
29
Level of Agreement
Strong (clinically acceptable) 0.75
Moderate (questionable value) 0.40.75
Poor (not clinically acceptable) 0.40
Table 3. Example Comparing Response Criterion 3 With
Response Criterion 13
Criterion 13
Responders Nonresponders Sum
Criterion 3
Responders 101 61 162
Nonresponders 60 39 99
Sum 161 100 261
Example comparing response criterion 3 (decrease in LVESV 10%, no
death due to heart failure) with response criterion 13 (increase in 6-minute
walk distance 10%, no death due to heart failure, no transplant) shows that
0.02 based on Equations 1 and 2. This -value of 0.02 demonstrates poor
agreement between the 2 criteria despite the fact that each criterion had nearly
identical response rates of approximately 62%.
Table 4. Response Rates for the Different Criteria
Response Criteria
Response
Rate, %
No. Evaluable
(% of Total)
Echocardiographic
1LVEF 5 units 51 286 (67)
1LVEF 15% (relative) 54 286 (67)
2LVESV 10%, no HF death 62 291 (68)
2LVESV 15% 56 286 (67)
LVESV 115% of baseline 91 286 (67)
2LVEDV 15% 49 286 (67)
1Stroke volume 15% 34 286 (67)
Clinical
2NYHA 1 71 385 (90)
2NYHA 1, no HF death 70 390 (92)
2NYHA 1 and 16MWD 25% 33 348 (82)
2NYHA 1 and 16MWD 25%, no HF
death
32 353 (83)
16MWD 10%, no HF death, no transplant 61 353 (83)
Two of the following 3: 2NYHA 1, 16MWD
50 m, 2QOL 15
63 339 (80)
Clinical composite score improved 69 426 (100)
Combined
1LVEF 5 units or 16MWD 50 m and
2NYHA 1 or 2QOL 10
71 250 (59)
Abbreviations as in Table 1.
Fornwalt et al Poor Agreement Among CRT Response Criteria 1987
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
range0.14 to 0.97). Agreement between echocardiographic
and clinical criteria was poor (Figure 4; mean 0.050.05,
median0.04, range0.03 to 0.17), with all 49 -values
showing poor agreement. The agreement among the echocar-
diographic parameters was not significantly different from
the agreement among the clinical parameters (uncorrected
P0.35). The response criterion based on a combination of
both echocardiographic and clinical measures showed signif-
icantly better agreement (P0.003) with clinical response
criteria (0.440.10) than with echocardiographic response
criteria (0.210.12). Bootstrap resampling of the -statistic
comparing clinical composite response and a 15% reduction
in LVESV justified that the sample size was large enough to
estimate the -value (Kolmogorov-Smirnov test for normality
P0.15).
Subgroup Analysis Excluding Response Criteria
Quantified Short Term and at 3 Months
Exclusion of the short-term and 3-month response criteria did
not significantly affect the results. After exclusion of 1
short-term (criterion 8 in Table 1) and two 3-month (criteria
15 and 17 in Table 1) response measures, the agreement
among the 12 remaining response criteria was poor as a group
(mean 0.220.26, median0.14, range0.03 to 0.97).
Agreement among the 6 remaining echocardiographic re-
sponse criteria was moderate (mean 0.440.24, medi-
an0.47, range0.16 to 0.88). Agreement among the 6
remaining clinical response criteria was also moderate (mean
0.420.27, median0.32, range0.14 to 0.97). Finally,
agreement between echocardiographic and clinical criteria
remained poor (mean 0.050.05, median0.04,
range0.03 to 0.17).
Discussion
The major findings of this study are as follows: (1) The 26
most-cited publications on predicting response to CRT used
17 different primary response criteria, and the level of
agreement not due to chance among 15 of these response
criteria was poor 75%of the time and strong only 4%of the time
Figure 2. Agreement among the 15
response criteria was poor. The -axis
shows the following ranges delineated
by dotted lines: strong agreement
(0.75), moderate agreement
(0.40.75), and poor agreement
(0.4). The worst agreement was
between echocardiographic (Echo) and
clinical (Clin) parameters. *P0.001 vs
Echo vs Echo and Clin vs Clin.
Figure 3. Agreement among the 15
response criteria was classied as poor for
75% of the 105 possible comparisons.
-Values are color-coded according to the
following ranges: greenstrong agreement
(0.75), yellowmoderate agreement
(0.40.75), and redpoor agreement
(0.4). Echo indicates echocardiographic
response criteria; Clin, clinical response
criteria.
1988 Circulation May 11, 2010
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
in the 426 patients enrolled in the PROSPECT study; (2)
agreement between echocardiographic and clinical response
criteria was poor and nearly equal to the level of agreement
expected by chance; (3) the percentage of patients defined as
having a positive response to CRT ranged from 32% to 91% for
the 15 response criteria; and (4) 99% of patients were classified
as a responder by at least 1 of the 15 criteria, whereas 94% were
classified as a nonresponder by at least 1 criterion.
Comparison to the Literature
To the best of our knowledge, no study has quantified
agreement among response criteria with a -coefficient;
however, a recent study by Bleeker et al
2
aimed to quantify
the agreement between echocardiographic and clinical
measures of response to CRT. The authors compared a
decline in New York Heart Association class (clinical
response) with a 15% decrease in LVESV (echocardio-
graphic response) in 144 consecutive patients undergoing
CRT. The authors concluded that the agreement between
[clinical response and echocardiographic response] was
good based on the observed agreement of 76%. However,
their data show that clinical and echocardiographic re-
sponses would be expected to agree 52% of the time on the
basis of chance alone. The study did not calculate -values
to account for this expected level of agreement due to
chance. We estimated the -value to be 0.50 from their
data, which is higher than the value of 0.17 observed in the
present study. However, the main conclusion that should
be drawn from both studies is similar: The agreement
between echocardiographic and clinical criteria for defin-
ing a positive response to CRT is only slightly better than
that expected by chance alone.
In the MIRACLE trial (Multicenter InSync Randomized
Clinical Evaluation), correlation between the change in left
ventricular end-diastolic volume and change in New York
Heart Association class after 6 months of CRT was weak
(r0.13).
30
In addition, the correlation between the change
in distance walked in 6 minutes and change in left
ventricular ejection fraction was weak (r0.15).
30
These
data are consistent with the present results, which show
poor agreement between clinical and echocardiographic
response criteria.
Previous studies have reported different rates of re-
sponse to CRT when different definitions of response are
used within the same population. For example, the PROS-
PECT study reported that 56% of patients were echocar-
diographic responders (defined by a reduction in LVESV
of at least 15%), whereas 69% of patients were clinical
responders (defined by an improvement in the clinical
composite score).
10
Thus, one would expect these mea-
sures to show poor agreement because of the different
response rates. However, the actual response rate does not
tell the entire story: Table 3 shows 2 different response
criteria with identical response rates of 62%, and despite
the identical response rates, the criteria show very poor
agreement (0.02). Thus, assessment of agreement with
the -statistic provides valuable information in addition to
the overall response rate of the population.
Figure 4. Agreement among the response criteria was poor 75% of the time and strong only 4% of the time. -Values are color-coded
according to the following ranges: Greenstrong agreement (0.75), yellowmoderate agreement (0.40.75), and redpoor
agreement (0.4). LVEF indicates left ventricular ejection fraction; HF, heart failure; NYHA, New York Heart Association; LVESV, left
ventricular end-systolic volume; LVEDV, left ventricular end-diastolic volume; 6MWD, distance walked in 6 minutes; and QOL, quality-
of-life score.
Fornwalt et al Poor Agreement Among CRT Response Criteria 1989
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
Other Inconsistencies in Defining Response
to CRT
Length of Follow-Up
Another area of inconsistency in defining response to CRT is the
length of the follow-up period after which a patient is deemed
either a responder or a nonresponder. Some studies focused on
short-term (1 to 2 days) response,
1,4,21,22
whereas most focused
on 3-month
59,18
or 6-month
24,1017,20,2327
response. CRT has
been shown to have persistent, increasing benefits with a longer
mean follow-up period of 29.5 months.
31
We defined response at
6 months because this was the prespecified follow-up period for
the PROSPECT study. We also performed a subgroup analysis
after excluding criteria that were assessed in the literature at
short-term and 3-month follow-up only, and this did not change
the present results. Future studies will be needed to address
agreement among the different lengths of follow-up.
Mortality
Whether death should be considered a nonresponse to CRT is
another area in which there is inconsistency. There are at least 3
different methods that authors have used to incorporate death
into their response criteria: (1) Death due to worsening heart
failure is included in the nonresponder group,
11,16,17,20,23,27
(2)
death due to any cause is included in the nonresponder group,
24
and (3) deaths are excluded from analysis.
3,5,6,8,9,26
Moreover,
numerous publications fail to specify how death was incorpo-
rated into response criteria despite enrolling consecutive patients
and following them for a 3- to 6-month period.
2,1214,18
Although
inclusion of all-cause mortality as a criterion for nonresponse
may not be appropriate, a patient who dies of progressive heart
failure should, objectively, be classified as a nonresponder.
Regardless, there is no consistent method for incorporating
mortality into the definition of response to CRT, and this needs
to be standardized.
A Consensus Definition of Response to CRT
Because heart failure is a debilitating life-threatening disease,
an effective heart failure therapy should treat both symptoms
and quality and duration of life.
32
Thus, measures of re-
sponse to CRT should either directly measure outcomes or
have a surrogate relationship with benefits in heart failure
symptoms, quality of life, and duration of life. The clinical
composite score
33
is a measure of response that accounts for
all of these factors and may be the best overall choice for
defining response in future CRT trials.
Study Limitations
The results of the present study are limited by the fact that we
used data from a single study (PROSPECT). However,
PROSPECT was a multicenter study that enrolled 457 wide-
QRS patients from 53 different centers across Europe, Hong
Kong, and the United States. We would expect similar results
from other large, multicenter databases.
The present results show that many different methods to
define a positive response to CRT are being used in the
literature and show poor agreement among each other. This
begs the question, which method should we use in the future
to determine whether a patient benefited from CRT? The
present study did not attempt to address this question, and
future studies will need to explore this important issue.
The definition of clinically acceptable agreement based on the
-coefficient is not standardized. Fleiss
29
proposed a threshold of
0.75 to define strong evidence for agreement that is not due to
chance, which is what we used; however, this threshold is
somewhat arbitrary. Landis and Koch
34
proposed that any value
of above 0.60 suggests substantial agreement, and 0.80
implies almost perfect agreement. However, the use of a
different threshold, such as 0.6, to define strong agreement
would not significantly alter the present results; 4 of the 105
-statistics that we calculated were greater than 0.8, and only 8
were 0.6. Thus, regardless of the threshold used, we observed
mostly poor agreement among the 15 different response criteria.
Conclusions
The 26 most-cited publications on predicting response to
CRT define response using 17 different criteria. Agreement
between these different published methods to define response
to CRT is poor 75% of the time and strong only 4% of the
time. This inconsistency in the definition of response to CRT
severely limits the ability to generalize results over multiple
studies and hinders progress in the field.
Sources of Funding
This work was supported by grants from the American Heart Associa-
tion (Grant-in-Aid No. 0855386E) and the National Institutes of Health
(HL089160) to Dr Oshinski. Dr Fornwalt was supported in part by
National Institutes of Health training grant No. 5 T32 GM008169.
Disclosures
Dr Gerritse is an employee of Medtronic Inc and owns company
stock. The remaining authors report no conflicts.
References
1. Bax JJ, Marwick TH, Molhoek SG, Bleeker GB, van Erven L, Boersma
E, Steendijk P, van der Wall EE, Schalij MJ. Left ventricular dyssyn-
chrony predicts benefit of cardiac resynchronization therapy in patients
with end-stage heart failure before pacemaker implantation. Am J
Cardiol. 2003;92:12381240.
2. Bleeker GB, Bax JJ, Fung JWH, van der Wall EE, Zhang Q, Schalij MJ, Chan
JYS, Yu CM. Clinical versus echocardiographic parameters to assess response to
cardiac resynchronization therapy. Am J Cardiol. 2006;97:260263.
3. Gorcsan J, Tanabe M, Bleeker GB, Suffoletto MS, Thomas NC, Saba S,
Tops LF, Schalij MJ, Bax JJ. Combined longitudinal and radial dyssyn-
chrony predicts ventricular response after resynchronization therapy.
J Am Coll Cardiol. 2007;50:14761483.
4. Suffoletto MS, Dohi K, Cannesson M, Saba S, Gorcsan J. Novel speckle-
tracking radial strain from routine black-and-white echocardiographic
images to quantify dyssynchrony and predict response to cardiac resyn-
chronization therapy. Circulation. 2006;113:960968.
5. Notabartolo D, Merlino JD, Smith AL, DeLurgio DB, Vera FV, Easley
KA, Martin RP, Leon AR. Usefulness of the peak velocity difference by
tissue Doppler imaging technique as an effective predictor of response to
cardiac resynchronization therapy. Am J Cardiol. 2004;94:817820.
6. Yu CM, Chan YS, Zhang Q, Yip GWK, Chan CK, Kum LCC, Wu L, Lee
APW, Lam YY, Fung JWH. Benefits of cardiac resynchronization therapy
for heart failure patients with narrow QRS complexes and coexisting systolic
asynchrony by echocardiography. J Am Coll Cardiol. 2006;48:22512257.
7. Yu CM, Fung JWH, Chan CK, Chan YS, Zhang Q, Lin H, Yip GWK,
Kum LCC, Kong SL, Zhang Y, Sanderson JE. Comparison of efficacy of
reverse remodeling and clinical improvement for relatively narrow and
wide QRS complexes after cardiac resynchronization therapy for heart
failure. J Cardiovasc Electrophysiol. 2004;15:10581065.
8. Yu CM, Zhang Q, Chan YS, Chan CK, Yip GWK, Kum LCC, Wu EB,
Lee PW, Lam YY, Chan S, Fung JWH. Tissue Doppler velocity is
superior to displacement and strain mapping in predicting left ventricular
1990 Circulation May 11, 2010
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from
reverse remodelling response after cardiac resynchronisation therapy.
Heart. 2006;92:14521456.
9. Yu CM, Zhang Q, Fung JWH, Chan HCK, Chan YS, Yip GWK, Kong
SL, Lin H, Zhang Y, Sanderson JE. A novel tool to assess systolic
asynchrony and identify responders of cardiac resynchronization therapy
by tissue synchronization imaging. J Am Coll Cardiol. 2005;45:677684.
10. Chung ES, Leon AR, Tavazzi L, Sun JP, Nihoyannopoulos P, Merlino J,
Abraham WT, Ghio S, Leclercq C, Bax JJ, Yu CM, Gorcsan J, Sutton
MS, De Sutter J, Murillo J. Results of the Predictors of Response to CRT
(PROSPECT) trial. Circulation. 2008;117:26082616.
11. Diaz-Infante E, Mont L, Leal J, Garcia-Bolao I, Fernandez-Lozano I,
Hernandez-Madrid A, Perez-Castellano N, Sitges M, Pavon-Jimenez R,
Barba J, Cavero MA, Moya JL, Perez-Isla L, Brugada J; SCARS Inves-
tigators. Predictors of lack of response to resynchronization therapy. Am J
Cardiol. 2005;95:14361440.
12. Molhoek SG, Bax JJ, Bleeker GB, Boersma E, van Erven L, Steendijk P,
van der Wall EE. Comparison of response to cardiac resynchronization
therapy in patients with sinus rhythm versus chronic atrial fibrillation.
Am J Cardiol. 2004;94:15061509.
13. Molhoek SG, Bax JJ, Boersma E, Van Erven L, Bootsma M, Steendijk P,
Van Der Wall EE, Schalij MJ. QRS duration and shortening to predict
clinical response to cardiac resynchronization therapy in patients with
end-stage heart failure. Pacing Clin Electrophysiol. 2004;27:308313.
14. Molhoek SG, Bax JJ, van Erven L, Bootsma M, Boersma E, Steendijk P, van
der Wall EE, Schalij MJ. Comparison of benefits from cardiac resynchroni-
zation therapy in patients with ischemic cardiomyopathy versus idiopathic
dilated cardiomyopathy. Am J Cardiol. 2004;93:860863.
15. Bax JJ, Bleeker GB, Marwick TH, Molhoek SG, Boersma E, Steendijk P,
van der Wall EE, Schalij MJ. Left ventricular dyssynchrony predicts
response and prognosis after cardiac resynchronization therapy. J Am Coll
Cardiol. 2004;44:18341840.
16. Bleeker GB, Kaandorp TAM, Lamb HJ, Boersma E, Steendijk P, de Roos
A, van der Wall EE, Schalij MJ, Bax JJ. Effect of posterolateral scar
tissue on clinical and echocardiographic improvement after cardiac resyn-
chronization therapy. Circulation. 2006;113:969976.
17. Ypenburg C, Schalij MJ, Bleeker GB, Steendijk P, Boersma E, Dibbets-
Schneider P, Stokkel MPM, van der Wall EE, Bax JJ. Impact of viability
and scar tissue on response to cardiac resynchronization therapy in
ischaemic heart failure patients. Eur Heart J. 2007;28:3341.
18. White JA, Yee R, Yuan XP, Krahn A, Skanes A, Parker M, Klein G,
Drangova M. Delayed enhancement magnetic resonance imaging predicts
response to cardiac resynchronization therapy in patients with intraven-
tricular dyssynchrony. J Am Coll Cardiol. 2006;48:19531960.
19. Thomson Reuters. Web of Science, Science Citation Index Expanded
database; ISI Web of Knowledge. Available at: http://apps.isiknowledge.
com. Available at: http://apps.isiknowledge.com. Accessed June 15, 2009.
20. Bleeker GB, Mollema SA, Holman ER, Van De Veire N, Ypenburg C,
Boersma E, van der Wall EE, Schalij MJ, Bax JJ. Left ventricular
resynchronization is mandatory for response to cardiac resynchronization
therapy: analysis in patients with echocardiographic evidence of left
ventricular dyssynchrony at baseline. Circulation. 2007;116:14401448.
21. Dohi K, Suffoletto MS, Schwartzman D, Ganz L, Pinsky MR, Gorcsan J.
Utility of echocardiographic radial strain imaging to quantify left ven-
tricular dyssynchrony and predict acute response to cardiac resynchroni-
zation therapy. Am J Cardiol. 2005;96:112116.
22. Gorcsan J, Kanzaki H, Bazaz R, Dohi K, Schwartzman D. Usefulness of
echocardiographic tissue synchronization imaging to predict acute response
to cardiac resynchronization therapy. Am J Cardiol. 2004;93:11781181.
23. Henneman MM, Chen J, Dibbets-Schneider P, Stokkel MR, Bleeker GB,
Ypenburg C, van der Wall EE, Schalij MJ, Garcia EV, Bax JJ. Can LV
dyssynchrony as assessed with phase analysis on gated myocardial perfusion
SPECT predict response to CRT? J Nucl Med. 2007;48:11041111.
24. Lecoq G, Leclercq C, Leray E, Crocq C, Alonso C, de Place C, Mabo P,
Daubert C. Clinical and electrocardiographic predictors of a positive
response to cardiac resynchronization therapy in advanced heart failure.
Eur Heart J. 2005;26:10941100.
25. Marcus GM, Rose E, Viloria EM, Schafer J, De Marco T, Saxon LA,
Foster E; VENTAK CHF/CONTAK-CD Biventricular Pacing Study
Investigators. Septal to posterior wall motion delay fails to predict reverse
remodeling or clinical improvement in patients undergoing cardiac resyn-
chronization therapy. J Am Coll Cardiol. 2005;46:22082214.
26. Stellbrink C, Breithardt OA, Franke A, Sack S, Bakker P, Auricchio A,
Pochet T, Salo R, Kramer A, Spinelli J; CPI Guidant Congestive Heart
Failure Research Group. Impact of cardiac resynchronization therapy
using hemodynamically optimized pacing on left ventricular remodeling
in patients with congestive heart failure and ventricular conduction dis-
turbances. J Am Coll Cardiol. 2001;38:19571965.
27. Ypenburg C, Roes SD, Bleeker GB, Kaandorp TAM, de Roos A, Schalij
MJ, van der Wall EE, Bax JJ. Effect of total scar burden on contrast-
enhanced magnetic resonance imaging on response to cardiac resynchro-
nization therapy. Am J Cardiol. 2007;99:657660.
28. Hulley SB, Cummings SR, Browner WS, Grady D, Hearst N, Newman
TB. Designing Clinical Research: An Epidemiologic Approach. 2nd ed.
Philadelphia, Pa: Lippincott Williams & Wilkins; 2001.
29. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New
York, NY: Wiley; 1981.
30. St John Sutton MG, Plappert T, Abraham WT, Smith AL, DeLurgio DB,
Leon AR, Loh E, Kocovic DZ, Fisher WG, Ellestad M, Messenger J,
Kruger K, Hilpisch KE, Hill MR. Effect of cardiac resynchronization
therapy on left ventricular size and function in chronic heart failure.
Circulation. 2003;107:19851990.
31. Cleland JG, Daubert JC, Erdmann E, Freemantle N, Gras D, Kappenberger
L, Tavazzi L. Longer-term effects of cardiac resynchronization therapy on
mortality in heart failure [the CArdiac REsynchronization-Heart Failure
(CARE-HF) trial extension phase]. Eur Heart J. 2006;27:19281932.
32. Anand IS, Florea VG, Fisher L. Surrogate end points in heart failure.
J Am Coll Cardiol. 2002;39:14141421.
33. Packer M. Proposal for a new clinical end point to evaluate the efficacy
of drugs and devices in the treatment of chronic heart failure. J Card Fail.
2001;7:176182.
34. Landis JR, Koch GG. The measurement of observer agreement for cate-
gorical data. Biometrics. 1977;33:159174.
CLINICAL PERSPECTIVE
A literature search revealed that the 26 most-cited publications on predicting response to cardiac resynchronization therapy
defined response using 17 different criteria. No study has investigated agreement among these various response criteria, and we
hypothesized that this agreement would be poor. The agreement among 15 of the 17 response criteria was assessed in 426 patients
from the PROSPECT study using the Cohen -coefficient (2 of the 17 response criteria were not calculable from PROSPECT
data). Response rates for the entire population were highly varied and ranged from 32% to 91% for the 15 criteria. Ninety-nine
percent of patients showed a positive response by at least 1 of the 15 criteria, whereas 94% were classified as a nonresponder
by at least 1 criterion. -Values were calculated for all 105 possible comparisons among the 15 response criteria and classified
into standard ranges: Poor agreement (0.4), moderate agreement (0.40.75), and strong agreement (0.75).
Seventy-five percent of the comparisons showed poor agreement, 21% showed moderate agreement, and only 4% showed strong
agreement. Thus, agreement between different methods to define response to cardiac resynchronization therapy is poor 75% of
the time and strong only 4% of the time, which severely impairs the ability to generalize results over multiple studies. This lack
of standardization hinders progress in cardiac resynchronization therapy research and needs to be resolved.
Go to http://cme.ahajournals.org to take the CME quiz for this article.
Fornwalt et al Poor Agreement Among CRT Response Criteria 1991
by guest on July 5, 2014 http://circ.ahajournals.org/ Downloaded from