Including stroke severity in risk models associated with improved prediction of risk of death

CHICAGO – Adding stroke severity to a hospital 30-day mortality model based on claims data for Medicare beneficiaries with acute ischemic stroke was associated with improvement in predicting the risk of death at 30 days and changes in performance ranking regarding mortality for a considerable proportion of hospitals, according to a study in the July 18 issue of JAMA.

"Increasing attention has been given to defining the quality and value of health care through reporting of process and outcome measures. National quality profiling efforts have begun to report hospital-level performance for Medicare beneficiaries, including 30-day mortality rates, for common medical conditions, including acute myocardial infarction [heart attack], heart failure, and community-acquired pneumonia," according to background information in the article. Stroke is among the leading causes of death, disability, hospitalizations, and health care expenditures in the United States. "There is increasing interest in reporting risk-standardized outcomes for Medicare beneficiaries hospitalized with acute ischemic stroke, but whether it is necessary to include adjustment for initial stroke severity has not been well studied."

Gregg C. Fonarow, M.D., of the University of California, Los Angeles, and colleagues conducted a study to evaluate the degree to which hospital outcome ratings and the ability to predict 30-day mortality are altered after including initial stroke severity in a claims-based risk model for hospital 30-day mortality for acute ischemic stroke. For the study, data were analyzed from 782 Get With The Guidelines-Stroke participating hospitals on 127,950 fee-for-service Medicare beneficiaries with ischemic stroke. The patients had a score documented for the National Institutes of Health Stroke Scale (NIHSS, a 15-item neurological examination scale with scores from 0 to 42, with higher scores indicating more severe stroke) between April 2003 and December 2009. The median (midpoint) age was 80 years, 57 percent were women, and 86 percent were white. Performance of claims-based hospital mortality risk models with and without inclusion of NIHSS scores for 30-day mortality was evaluated and hospital rankings from both models were compared. The NIHSS median score in this overall population was 5, and the median hospital-level NIHSS score was 5.

There were 18,186 deaths (14.5 percent) within the first 30 days, including 7,430 deaths during the index hospitalization (in-hospital mortality, 5.8 percent). The median hospital-level 30-day mortality rate was 14.5 percent. The researchers found that the hospital mortality model with NIHSS scores had significantly better discrimination than the model without. Also, other index scores demonstrated substantially more accurate classification of hospital 30-day mortality after the addition of NIHSS score to the claims model. The model with NIHSS exhibited better agreement between observed and predicted mortality rates.

Analysis of data indicated that more than 40 percent of hospitals identified in the top or bottom 5 percent of hospital risk-adjusted mortality would have been reclassified into the middle mortality range using a model adjusting for NIHSS score compared with a model without NIHSS score adjustment. "Similarly, when considering the top 20 percent and bottom 20 percent ranked hospitals, close to one-third of hospitals would have been reclassified," the authors write.

"These findings highlight the importance of including a valid specific measure of stroke severity in hospital risk models for mortality after acute ischemic stroke for Medicare beneficiaries. Furthermore, this study suggests that inclusion of admission stroke severity may be essential for optimal ranking of hospital with respect to 30-day mortality."

"As public reporting and value-based purchasing policies increase for outcome measures, it is important to recognize the effect that using models with less than ideal discrimination and calibration has on the ranking of hospitals and the lack of correlation among ranking by models that do and do not adjust for critical risk determinants," the researchers write.

(JAMA. 2012;308[3]:257-264. Available pre-embargo to the media at http://media.jamanetwork.com)

Editor's Note: Please see the article for additional information, including other authors, author contributions and affiliations, financial disclosures, funding and support, etc.

In an accompanying editorial, Tobias Kurth, M.D., Sc.D., of the University of Bordeaux, France, and Mitchell S. V. Elkind, M.D., M.S., of Columbia University, New York, write that the results of this study "clearly highlight the importance of incorporating information on stroke severity when conducting health outcomes research in stroke."

"Excluding this information will lead to incorrect ranking of hospital performance by failing to consider that hospitals care for different patient populations. The influence of stroke severity on these outcome measures, moreover, seems different from that of measures of severity in other conditions. For other cardiovascular diseases, risk adjustment using demographic characteristics and claims-derived comorbid conditions may sufficiently account for the underlying case mix. Ischemic stroke is a much more heterogeneous condition than ischemic heart disease and is characterized by multiple subtypes, etiologies, and diverse outcomes. The assumption that what is true of myocardial infarction is also true of stroke, therefore, is flawed, as the present data underscore. The particular characteristics of stroke have to be taken into consideration by clinicians, insurance companies, and policy makers when comparing disease-specific health outcomes."

(JAMA. 2012;308[3]:292-293. Available pre-embargo to the media at http://media.jamanetwork.com)

Editor's Note: Please see the article for additional information, including financial disclosures, funding and support, etc.

To contact Gregg C. Fonarow, M.D., call Rachel Champeau at 310-794-2270 or email RChampeau@mednet.ucla.edu. To contact editorial co-author Tobias Kurth, M.D., Sc.D., email tobias.kurth@univ-bordeaux.fr.