News from the AAN Annual Meeting
Neurologist Intuition vs. Screening Scales for ICH Prognosis — Which is More Accurate?
ARTICLE IN BRIEF
Doctors who factored comorbidities into their predictions of outcomes for patients admitted with intracerebral hemorrhage (ICH) tended to be more accurate than two commonly used clinical grading scales for ICH.
WASHINGTON—When it comes to forecasting patients' functional outcome three months down the line following hospitalization for intracerebral hemorrhage (ICH), doctors' gut predictions in the first 24 hours after admission proved more accurate than either the ICH or FUNC scores, two commonly used clinical grading scales for predicting outcomes. Moreover, the accurate physician predictions were twice as likely to have factored in patient comorbidities, according to a study presented here in April at the AAN Annual Meeting.
David Y. Hwang, MD, an assistant professor of neurology at the Yale School of Medicine and a neurointensivist at Yale-New Haven Hospital, told Neurology Today that he and his colleagues wanted to determine whether predictions that incorporated variables outside those comprising the standard ICH and FUNC scores would be more accurate than predictions that did not.
Dr. Hwang and his colleagues found that physicians were more accurate than the ICH scores at predicting outcomes, even when combining doctors who factored in comorbidities and those who did not into one cohort.
“Regardless of whatever factors folks are using, comorbidities or not, physicians making gut predictions are more accurate than the existing scales,” he said.
What was their one consistent failing? The data suggest that physicians were not as accurate in predicting mortality outcomes, Dr. Hwang said. After 90 days, 35.5 percent of the 121 patients had died; the physicians had predicted that only 14.1 percent of them would be dead at 90 days.
For the study, physicians who treated ICH were asked to predict the modified Rankin Scale (mRS) score at three months for 121 patients with primary ICH who were admitted consecutively to five centers. (The mRS is a scale ranging from 0 to 6, where 0 means no disability and 6 indicates death.) All predictions were collected within 24 hours of admission. Physicians were also asked to indicate up to 10 factors influencing their prediction, whatever they might be. The survey did not specifically ask the physicians to list patients' comorbidities.
Comparing the accuracy of the physicians' predictions to that of the ICH Score was tricky. The ICH Score output is a value from 0 to 6, where patients with a score of 6 tend to have poorer outcomes than 5 at three months, 5 having poorer outcomes than 4, and so on. “Perfection” for the ICH Score would mean that every patient with a score of 6 did worse than everyone with a 5, and so on. But physicians' accuracy was measured by whether or not they could precisely predict the mRS score at 90 days.
To compare the accuracy of the two methods, Dr. Hwang's team computed a Spearman's rank correlation coefficient (r) for both, with a perfect coefficient being 1.
In the cohort of 121 patients, the r for the ICH Score with regard to three-month outcome was 0.62, whereas the r for the physician predictions was 0.75 (p=0.057). “The statistical scatter for the physician predictions compared to the actual outcomes was, as a whole, less than that for the ICH Score,” Dr. Hwang said.
In all, Dr. Hwang's group collected 37 accurate and 84 inaccurate predictions. No significant differences were found between the patient groups regarding age, ICH volume, or general clinical exam on admission. However, 16 (43.2 percent) of the physicians who made accurate predictions listed patients' general comorbidities as a factor in prediction, compared with 20 (23.8 percent) among those who made inaccurate predictions (p=0.05).
Whether the scales should be modified to include comorbidities remains uncertain, Dr. Hwang said. “There's a cost-benefit to every scale. You can make your scale complicated to make it more accurate, but then you lose the spirit of what the scales were designed to do in the first place. Scales that have staying power are quite simple, and the ICH Score is one of them.”
Commenting on the study, J. Claude Hemphill III, MD, FAAN, a professor of neurology and neurological surgery and chair of neurocritical care at the University of California, San Francisco, who led the development of the ICH scoring tool, said that the scale was never meant to replace physicians' judgment.
“The ICH Score was never intended to be used to precisely predict outcome,” said Dr. Hemphill, who was not involved with the current study. “Clinicians should integrate knowledge of the literature with expert clinical judgment, not just use a scoring system. That's both evidence-based medicine and the art of being a physician.”
Dr. Hemphill said the the ICH scale was intended as a clinical grading tool that would improve communication and risk stratification in acute ICH. “Many folks just can't seem to help hanging their hat on numbers, even though there are many things that go into prognostication,” he said. He added, “I still think the ICH Score is a useful way to stratify risk. An ICH score of 4 is worse than 3, 3 is worse than 2, and this consistently holds up.”
Mitchell S. Elkind, MD, a professor of neurology and epidemiology at Columbia University, said the scales need to balance accuracy against simplicity. “You don't want a scale that is more complicated than it needs to be,” he said, “but if comorbidities are prognostic, they need to be included.”
In fact, a study led by Dr. Hemphill in 2011 did exactly that, by adjusting the ICH Score to include the Charlson Comorbidity Index (CCI), which accounts for multiple comorbidities by creating a sum score weighted according to the presence of various conditions.
“Future ICH outcome studies should account for the impact of comorbidities on patient outcome,” the study concluded.
The finding that comorbidities do matter is an important one, and comports with physicians' intuitions, Dr. Elkind said. “Even though they're not part of the score,” he said, “on the floors and in the ICUs when we take care of these patients, these other illnesses influence our sense of how sick a patient really is, and that influences our treatment. If we believe a patient has a very poor prognosis, some would argue, why put the patient and the family through the trauma of treatment — surgery to remove a blood clot from the brain, intubation, putting them on a respirator, having other lines and tubes inserted? An accurate prognosis does have implications for how we talk with the family, giving them information so they can make the most appropriate decision for their family member.”
EXPERTS: ON PHYSICIANS' PREDICTIONS OF ICH OUTCOMES COMPARED WITH ICH SCALES