Академический Документы
Профессиональный Документы
Культура Документы
preference for models that include hsCRP. With use of 10-year risk
categories of 0% to less than 5%, 5% to less than 10%, 10% to
less than 20%, and 20% or greater, risk prediction was more
accurate in models that included hsCRP, particularly for risk between 5% and 20%. Among women initially classified with risks of
5% to less than 10% and 10% to less than 20% according to the
Adult Treatment Panel III covariables, 21% and 19%, respectively,
were reclassified into more accurate risk categories. Although addition of hsCRP had minimal effect on the c-statistic (a measure of
model discrimination) once age, smoking, and blood pressure were
accounted for, the effect was nonetheless greater than that of total,
LDL, or HDL cholesterol, suggesting that the c-statistic may be
insensitive in evaluating risk prediction models.
Limitations: Data were available only for women.
Conclusions: A global risk prediction model that includes hsCRP
improves cardiovascular risk classification in women, particularly
among those with a 10-year risk of 5% to 20%. In models that
include age, blood pressure, and smoking status, hsCRP improves
prediction at least as much as do lipid measures.
www.annals.org
See also:
Print
Editors Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Editorial comment. . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Related article. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Summary for Patients. . . . . . . . . . . . . . . . . . . . . . . I-19
Web-Only
Appendix
Appendix Table
Conversion of figures and tables into slides
2006 American College of Physicians 21
Article
Contribution
The authors used the Womens Health Study, a nationwide cohort of 15 048 initially healthy women, to develop
a cardiovascular disease (CVD) risk prediction model using
hsCRP and Framingham risk model predictors. While
hsCRP improved overall model fit, the clinical utility of
hsCRP in terms of reclassification was most substantial for
those with a 5% or greater 10-year risk based on traditional risk factors.
Cautions
The study does not address the clinical value of lowering
hsCRP level.
Implications
In this largely low-risk population, adding hsCRP to the
Framingham model reclassified patients into groups that
better reflected their actual CVD risk. This effect was most
clinically relevant for those at intermediate risk.
The Editors
METHODS
We compared the clinical utility of global cardiovascular risk prediction models based on Framingham covariables with and without hsCRP among participants in the
Womens Health Study (WHS) (2325), a large-scale, nationwide cohort of U.S. women age 45 years and older
who were free of CVD and cancer at study entry. Women
were followed annually for the development of CVD, with
an average follow-up of 10 years. All reported CVD outcomes, including myocardial infarction, ischemic stroke,
coronary revascularization procedures, and deaths from
cardiovascular causes, were adjudicated by an end points
committee after medical record review. All study participants provided written informed consent, and the study
protocol was approved by the institutional review board of
Brigham and Womens Hospital in Boston, Massachusetts.
Baseline blood samples were assayed for C-reactive
protein with a validated, high-sensitivity assay (Denka
Seiken, Tokyo, Japan) and for total, high-density lipoprotein (HDL), and low-density lipoprotein (LDL) cholesterol
with direct-measurement assays (Roche Diagnostics, Basel,
Switzerland). Women who were diabetic at baseline were
excluded from predictive modeling because ATP III labeling considers diabetes to be a risk equivalent for coronary
heart disease (2). In parallel with guidelines established for
lipid evaluation (26), models were initially fitted in a derivation cohort limited to women not taking hormone replacement therapy at baseline (n ! 15 048 with data on all
variables) and were then applied to all nondiabetic women
(n ! 26 927) for clinical risk prediction.
22 4 July 2006 Annals of Internal Medicine Volume 145 Number 1
Article
Table 1. Best-Fitting Global Cardiovascular Risk Prediction Model among the Model Derivation Cohort of 15 048 Women from
the Womens Health Study*
Variable
WHS Model
!-Coefficient
Age
SBP # 125
(SBP # 125)2
Antihypertensive use
Current smoking
Ln(HDL)
Ln(Total cholesterol)
Ln(hsCRP)
Measures of fit
BIC
C-index
Calibration P value (risk percentage)
SE
0.074
0.032
#0.0003
0.264
0.965
#1.244
1.569
0.196
0.006
0.006
0.0002
0.123
0.117
0.205
0.269
0.050
With hsCRP
6960.26
0.815
0.23
Without hsCRP
6969.60
0.813
0.039
Hazard Ratio
P Value
1.08
1.03
1.00
1.30
2.62
0.29
4.80
1.22
"0.001
"0.001
0.058
0.032
"0.001
"0.001
"0.001
"0.001
* BIC ! Bayes information criterion; HDL ! high-density lipoprotein; hsCRP ! high-sensitivity C-reactive protein; Ln ! natural logarithm; SBP ! systolic blood pressure;
WHS ! Womens Health Study.
SBP is measured in mm Hg, HDL and total cholesterol are measured in mg/dL, and hsCRP is measured in mg/L.
Lower values of BIC and higher values of the c-index and calibration P value reflect better fit.
Risk estimates are provided on a natural log scale and were derived from
a Cox regression model using a flexible spline curve. Dotted lines represent 95% CIs.
4 July 2006 Annals of Internal Medicine Volume 145 Number 1 23
Article
Figure 2. Cardiovascular point scoring system for women based on Framingham covariables and high-sensitivity C-reactive protein
(hsCRP).
This scoring system is intended as an illustration only. CVD ! cardiovascular disease; HDL ! high-density lipoprotein; RR ! relative risk; SBP !
systolic blood pressure. To convert cholesterol values to mmol/L, multiply by 0.02586.
among women age 47 years and older at the 10th Framingham examination were evaluated. Mean HDL cholesterol
level at the 10th examination was estimated on the basis of
age and HDL cholesterol level at the 15th examination.
The population mean hsCRP value was estimated from a
24 4 July 2006 Annals of Internal Medicine Volume 145 Number 1
Article
RESULTS
Among the 15 048 women used for model development, the mean age was 54 years (SD, 8); 1841 women
(12%) were current smokers. A total of 2227 women
(15%) had blood pressure of 140/90 mm Hg or higher,
and 1802 women (12%) were taking antihypertensive
medication at baseline. Median lipid values were as follows:
total cholesterol level, 5.3 mmol/L (interquartile range, 4.7
to 6.1 mmol/L) [206 mg/dL (interquartile range, 181 to
234 mg/dL)]; LDL cholesterol level, 3.2 mmol/L (interquartile range, 2.6 to 3.8 mmol/L) [124 mg/dL (interquartile range, 102 to 147 mg/dL)]; and HDL cholesterol level,
1.3 mmol/L (interquartile range, 1.1 to 1.5 mmol/L) [49
mg/dL (interquartile range, 42 to 59 mg/dL)]. The median
hsCRP level was 1.5 mg/L (interquartile range, 0.6 to 3.4
mg/L). Over the mean 10-year follow-up, 390 women developed CVD, including 116 myocardial infarctions, 100
ischemic strokes, 217 coronary revascularization procedures, and 65 deaths due to cardiovascular cause. Some
women experienced more than 1 of these events.
The best-fitting global cardiovascular risk prediction
model in these data included all Framingham covariables
and hsCRP (Table 1). Figure 1 shows that after adjustment for all Framingham variables, there was a log-linear
relationship between hsCRP and risk for CVD. While risk
estimates based on the model coefficients given in Table 1
provide the best estimates, for clinical utility we also calculated a point scoring system (39) for cardiovascular risk
analogous to that used by the National Cholesterol Education Program (Figure 2). The absolute risk estimates in
Figure 2 are calibrated to the overall incidence in the Framingham Heart Study sample to enhance generalizability.
Relative Contributions to Global Risk of Age, Blood
Pressure, Smoking, Lipids, and hsCRP
Table 2 presents a comparison of the relative contributions to global risk made by each individual Framingham covariate and by hsCRP. Age is the strongest predictor of risk in these data, leading to a high likelihood ratio
and a c-index of 0.73. After adjustment for age, likelihood
ratio statistics demonstrated the strongest improvement in
fit for systolic blood pressure, followed in descending order
by hsCRP; current smoking; and HDL, total, and LDL
cholesterol. In models that included age, systolic blood
www.annals.org
Variable
Variable LR
Chi-Square
C-Index
Age only
$ SBP
$ Ln(hsCRP)
$ Current smoking
$ Ln(HDL cholesterol)
$ Ln(total cholesterol)
$ LDL cholesterol
100.60
86.72
75.04
70.19
36.72
31.13
267.05
367.66
353.77
342.10
337.24
303.78
298.18
0.731
0.768
0.763
0.757
0.765
0.747
0.746
44.05
41.89
26.28
22.94
444.59
488.64
486.48
470.87
467.53
0.791
0.800
0.801
0.796
0.796
For both the ATP III model and the final WHS
model, the likelihood ratio test for the inclusion of hsCRP
was highly significant (P " 0.001). Of note, the Bayes
information criterion indicated a strong preference for the
inclusion of hsCRP (Table 1) after adjustment for adding
a variable. This suggests that the model including hsCRP
provided better fit, even after adjustment for the increase in
number of predictors. Similarly, models that included
hsCRP demonstrated better calibration (higher P value for
calibration), while models without hsCRP had larger deviations between the observed and predicted probabilities in
the higher-risk categories (Figure 3). By contrast, the c4 July 2006 Annals of Internal Medicine Volume 145 Number 1 25
Article
Figure 3. Calibration curves for risk prediction models without (top) and with (bottom) high-sensitivity C-reactive protein (hsCRP)
in the model.
The model that includes hsCRP shows closer agreement between observed and model-based predicted risk. WHS ! Womens Health Study.
Article
hsCRP, and the Bayes information criterion weights, corresponding to the posterior model probabilities, strongly
supported the addition of hsCRP. As also shown in these
data, the relative contribution to global risk made by
hsCRP was at least as large as that made by total, HDL, or
LDL cholesterol. The added predictive value of including
hsCRP was most evident among those at 5% or greater
10-year risk.
While all the likelihood-based measures of fit improved when hsCRP was added, the c-index, or generalized
c-statistic, changed little with the addition of any bloodbased risk factor (including total, HDL, and LDL cholesterol) once we accounted for age, smoking, and blood pressure. In fact, the increase in the c-index associated with
hsCRP was greater than that for any of the lipids. This
observation underscores the limitations of the c-index, or
area under the receiver-operating characteristic curve, as a
method for determining model fit, despite its continued
popular use in the medical literature. The c-index is particularly suited to retrospective case control studies, in
which the actual outcome probabilities cannot be estimated (40). Since it is based exclusively on ranks, however,
it measures only how well the predicted values can rankorder the responses. It may not be as sensitive as the likelihood function in choosing between models, especially
when the models are strong (28). This may be particularly
true in settings where many individuals fall into low-risk
groups, as is true of cardiovascular risk detection in the
general population as well as in the WHS. In the current
DISCUSSION
The Framingham risk score provides a useful measure
of risk stratification for coronary heart disease and has been
valuable in clinical practice. Whether it can be improved
by including other simple and inexpensive measures has
not previously been determined. We fitted predictive models for major CVD, including myocardial infarction, coronary revascularization, stroke, and cardiovascular death, using traditional Framingham predictors, with and without
the addition of hsCRP. As shown, all global measures of
predictive accuracy were improved in prediction models
that included hsCRP. In particular, likelihood-based measures showed a strong preference for the models with
Table 3. Observed and Expected Risks among all 26 927 Nondiabetic Women in the Womens Health Study Using the Final
Global Risk Prediction Model with and without High-Sensitivity C-Reactive Protein*
10-Year Risk in WHS Model without hsCRP
0% to <5%
Total participants, n
Participants classified in each risk stratum
by the WHS model with hsCRP, %
10-y risk, %
5% to <10%
Total participants, n
Participants classified in each risk stratum
by the WHS model with hsCRP, %
10-y risk, %
10% to <20%
Total participants, n
Participants classified in each risk stratum
by the WHS model with hsCRP, %
10-y risk, %
>20%
Total participants, n
Participants classified in each risk stratum
by the WHS model with hsCRP, %
10-y risk, %
5% to <10%
10% to <20%
Total Reclassified
>20%
23 174
97.9
488
2.1
1.6
5.8
267
11.9
1773
78.7
213
9.5
0
0.0
2.4
7.8
15.2
0
0.0
110
13.7
653
81.3
40
5.0
6.8
11.5
19.9
0
0.0
0
0.0
30
14.4
179
85.7
18.8
27.1
0
0.0
0
0.0
2.1
21.3
18.7
14.4
* All estimated and observed risk estimates have been extrapolated to 10-y risk. hsCRP ! high-sensitivity C-reactive protein; WHS ! Womens Health Study.
www.annals.org
Article
data, the c-index could not distinguish between blood pressure, smoking, or any of the measured blood predictors,
even though the likelihood ratio statistics clearly ordered
variable contributions, with systolic blood pressure being
the strongest predictor after age. In fact, hsCRP was a
stronger individual predictor than any of the lipid measures in these data, including HDL cholesterol, its closest
competitor. Thus, reliance solely on the c-statistic for
model development could erroneously lead to the exclusion of lipids as well as hsCRP from risk prediction models.
Accuracy, or the predictive ability of a model, has 2
major components, discrimination and calibration (28).
The c-statistic is a measure of discrimination, or the ability
to separate 2 groups, such as case-patients and controls.
Particularly in a prospective study, calibration, or how well
the predicted probabilities reflect actual risk, is another
aspect of accuracy not captured by the c-statistic. A model
could discriminate well but lack even internal calibration if
the fitted scores do not reflect the true probability of an
event. The predicted probability given the risk factors, or
the post-test probability, can be more useful clinically in
assessing future cardiovascular risk than sensitivity or specificity, on which the c-statistic is based. Put another way, as
Moons and Harrell have stated (41), sensitivity and specificity have no direct diagnostic meaning; for the patient,
the issue is not the risk for having a positive test result, but
the risk for developing the disease. In our data, the model
that included hsCRP was better calibrated and was a better
predictor of the probability of disease.
Clinically, the inclusion of hsCRP in global prediction
models more accurately predicted true cardiovascular risk
in these data. While it did not strongly affect estimated risk
among women originally at very low risk, hsCRP had a
substantial effect among women at 5% or higher 10-year
risk. Among those classified as having 5% to less than 20%
risk, the addition of hsCRP reclassified about 20% of
women into more accurate risk strata. This would suggest
that an effective clinical strategy to improve risk prediction
might be to evaluate hsCRP among women with at least
5% predicted risk based on traditional risk factors.
One of the primary goals of any risk prediction algorithm for CVD is to identify individuals at increased risk
who will benefit from aggressive lifestyle changes, including dietary moderation, exercise, and smoking cessation, all
of which reduce hsCRP levels in addition to decreasing
cardiovascular risk. Thus, knowledge of hsCRP level and
more precise global risk estimation may help motivate patients to improve adherence to therapeutic lifestyle
changes, as currently advocated in the ATP III guidelines
(2). Ten-year risks are also commonly estimated to make
therapeutic decisions with regard to lipid-lowering therapy,
particularly with statin agents. Previous research demonstrates that statin agents decrease hsCRP levels in a manner
largely independent of LDL cholesterol, and the efficacy of
statin therapy is linked in part to underlying hsCRP levels
(42 46). As shown in our data, approximately 20% of
28 4 July 2006 Annals of Internal Medicine Volume 145 Number 1
References
1. Wilson PW, DAgostino RB, Levy D, Belanger AM, Silbershatz H, Kannel
WB. Prediction of coronary heart disease using risk factor categories. Circulation.
1998;97:1837-47. [PMID: 9603539]
www.annals.org
Article
to clinical and public health practice: a statement for healthcare professionals from
the Centers for Disease Control and Prevention and the American Heart Association. Circulation. 2003;107:499-511. [PMID: 12551878]
22. Grundy SM. The changing face of cardiovascular risk [Editorial]. J Am Coll
Cardiol. 2005;46:173-5. [PMID: 15992653]
23. Buring JE, Hennekens CH. The Womens Health Study: summary of the
study design. Journal of Myocardial Ischemia. 1992;4:27-9.
24. Rexrode KM, Lee IM, Cook NR, Hennekens CH, Buring JE. Baseline
characteristics of participants in the Womens Health Study. J Womens Health
Gend Based Med. 2000;9:19-27. [PMID: 10718501]
25. Ridker PM, Cook NR, Lee IM, Gordon D, Gaziano JM, Manson JE, et al.
A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med. 2005;352:1293-304. [PMID: 15753114]
26. Hainline A, Karon J, Lippel K, eds. Manual of Laboratory Operations, Lipid
Research Clinics Program, and Lipid and Lipoprotein Analysis. 2nd ed. Bethesda,
MD: Department of Health and Human Services; 1982.
27. Cox DR. Regression models and life tables (with discussion). Journal of the
Royal Statistical Society B. 1972;34:187-220.
28. Harrell FE Jr. Regression Modeling Strategies. New York: Springer; 2001.
29. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning:
Data Mining, Inference and Prediction. New York: Springer-Verlag; 2001.
30. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver
operating characteristic (ROC) curve. Radiology. 1982;143:29-36. [PMID:
7063747]
31. Efron B, Tibshirani R. An Introduction to the Bootstrap. New York: Chapman & Hall; 1993.
32. Burnham KP, Anderson DR. Model Selection and Inference: A Practical
Information-Theoretic Approach. New York: Springer-Verlag; 1998.
33. Nagelkerke NJ. A note on a general definition of the coefficient of determination. Biomka. 1991;78:691-2.
34. Ash A, Shwartz M. R2: a useful measure of model performance when predicting a dichotomous outcome. Stat Med. 1999;18:375-84. [PMID: 10070680]
35. Royston P, Sauerbrei W. A new measure of prognostic separation in survival
data. Stat Med. 2004;23:723-48. [PMID: 14981672]
36. Hosmer DW, Hosmer T, Le Cessie S, Lemeshow S. A comparison of
goodness-of-fit tests for the logistic regression model. Stat Med. 1997;16:965-80.
[PMID: 9160492]
37. Fleiss JL. Statistical Methods for Rates and Proportions. 2nd ed. New York:
Wiley; 1981.
38. Greenland P, Smith SC Jr, Grundy SM. Improving coronary heart disease
risk assessment in asymptomatic people: role of traditional risk factors and noninvasive cardiovascular tests. Circulation. 2001;104:1863-7. [PMID: 11591627]
39. Sullivan LM, Massaro JM, DAgostino RB Sr. Presentation of multivariate
data for clinical use: The Framingham Study risk score functions. Stat Med.
2004;23:1631-60. [PMID: 15122742]
40. Gail MH, Pfeiffer RM. On criteria for evaluating models of absolute risk.
Biostatistics. 2005;6:227-39. [PMID: 15772102]
41. Moons KG, Harrell FE. Sensitivity and specificity should be de-emphasized
in diagnostic accuracy studies. Acad Radiol. 2003;10:670-2. [PMID: 12809422]
42. Ridker PM, Rifai N, Pfeffer MA, Sacks FM, Moye LA, Goldman S, et al.
Inflammation, pravastatin, and the risk of coronary events after myocardial infarction in patients with average cholesterol levels. Cholesterol and Recurrent
Events (CARE) Investigators. Circulation. 1998;98:839-44. [PMID: 9738637]
43. Ridker PM, Rifai N, Clearfield M, Downs JR, Weis SE, Miles JS, et al.
Measurement of C-reactive protein for the targeting of statin therapy in the
primary prevention of acute coronary events. N Engl J Med. 2001;344:1959-65.
[PMID: 11430324]
44. Kent SM, Taylor AJ. Usefulness of lowering low-density lipoprotein cholesterol to "70 mg/dl and usefulness of C-reactive protein in patient selection. Am
J Cardiol. 2003;92:1224-7. [PMID: 14609606]
45. Ridker PM, Cannon CP, Morrow D, Rifai N, Rose LM, McCabe CH, et
al. C-reactive protein levels and outcomes after statin therapy. N Engl J Med.
2005;352:20-8. [PMID: 15635109]
46. Nissen SE, Tuzcu EM, Schoenhagen P, Crowe T, Sasiela WJ, Tsai J, et al.
Statin therapy, LDL cholesterol, C-reactive protein, and coronary artery disease.
N Engl J Med. 2005;352:29-38. [PMID: 15635110]
47. Boekholdt SM, Hack CE, Sandhu MS, Luben R, Bingham SA, Wareham
NJ, et al. C-reactive protein levels and coronary artery disease incidence and
mortality in apparently healthy men and women: The EPIC-Norfolk prospective
population study 1993-2003. Atherosclerosis. 2005. [PMID: 16257408]
4 July 2006 Annals of Internal Medicine Volume 145 Number 1 29
Preventive Medicine, Brigham and Womens Hospital, 900 Commonwealth Avenue East, Boston, MA 02215.
Author Contributions: Conception and design: N.R. Cook, P.M. Ridker.
APPENDIX
Computation of 10-Year Risk
The 10-year risk for cardiovascular disease calibrated to the
Framingham population may be estimated for each individual
woman using the !-coefficients in Table 1. First, multiply each
womans risk factor x by the appropriate coefficient in Table 1
and sum these (! "! # x). The risk may then be computed
from the following equation:
Risk ! 1 $ (0.903)exp("! # x
$8.795)
www.annals.org
Appendix Table. Comparison of Discrimination and Calibration for Global Risk Prediction Models with and without
High-Sensitivity C-Reactive Protein*
Variable
WHS Model
With hsCRP
Without hsCRP
With hsCRP
Without hsCRP
556.75 (8)
6960.26
0.874
6928.53
0.506
9.28
9.05
1.948
1.914
0.01960
541.44 (7)
6969.60
0.0082
6941.84
0.0007
9.03
8.92
1.922
1.893
0.01965
558.69 (9)
6964.28
0.117
6928.58
0.492
9.31
8.97
1.951
1.913
0.01959
542.54 (8)
6974.46
0.0007
6942.74
0.0004
9.05
8.84
1.918
1.884
0.01964
Discrimination
C-index
Adjusted c-index
0.815
0.813
0.813
0.811
0.814
0.810
0.812
0.809
Calibration
HosmerLemeshow statistic
P value, deciles
P value, risk percentage
0.19
0.23
0.59
0.039
0.71
0.25
0.79
0.008
Global measures
LR chi-square (df)
BIC
BIC weight
AIC
AIC weight
R2 , %
Adjusted R2, %
D-statistic
Adjusted D-statistic
Brier score
* Lower values of the BIC, AIC, and Brier score and higher values of all other statistics, including the calibration P values, indicate better fit. See text for descriptions. AIC !
Akaike information criterion; ATP ! Adult Treatment Panel; BIC ! Bayes information criterion; df ! degrees of freedom; hsCRP ! high-sensitivity C-reactive protein;
LR chi-square ! multi degree-of-freedom likelihood ratio chi-square statistic for fit of the entire model; WHS ! Womens Health Study.
Superior fit in comparison of models with and without hsCRP.
Adjusted for optimism using bootstrap resampling. Tests for improvement in the D-statistic with the addition of hsCRP to the ATP III model reached statistical
significance (P ! 0.03).
Based on 10 categories defined by 2percentage point increments in predicted risk.
www.annals.org