Assessment of Morbidity Over Time in Predicting Health Outcomes

Zeng, Chan PhD*; Ellis, Jennifer L. MSPH*; Steiner, John F. MD, MPH*; Shoup, Jo Ann MSW, MS*; McQuillan, Deanna B. MA*; Bayliss, Elizabeth A. MD, MSPH*,†

Medical Care:
doi: 10.1097/MLR.0000000000000033
Original Articles

Background: Administratively derived morbidity measures are often used in observational studies as predictors of outcomes. These typically reflect a limited time period before an index event; some outcomes may be affected by rate of morbidity change over longer preindex periods.

Objectives: The aim of the study was to develop statistical models representing the trajectory of individual morbidity over time and to evaluate the performance of trajectory versus other summary morbidity measures in predicting a range of health outcomes.

Methods: From a retrospective cohort study of integrated health system members aged 65 years or older with 3 or more common chronic medical conditions, we used available diagnoses for up to 10 years to examine associations between variations of the Charlson Comorbidity Index (CCI, Quan adaptation) and health outcomes. A linear mixed effects model was used to estimate the trajectory of individual CCI over time; estimated parameters describing individual trajectories were used as predictors for health outcomes. Other variations of CCI were: a “snapshot” measure, a cumulative measure, and actual baseline and rate of change. Models were developed in an initial cohort for whom we had survey data, and verified in a larger cohort.

Results: Among 961 surveyed members and 13,163 members of a secondary cohort, cumulative and snapshot measures provided best fit and predictive ability for utilization outcomes. Incorporating trajectory resulted in a slightly better model for self-reported health status.

Conclusions: Modeling longitudinal morbidity trajectories did not add substantially to the association between morbidity and utilization or mortality. Standard snapshot morbidity measures likely sufficiently capture multimorbidity in assessing these outcomes.

Author Information

*Institute for Health Research, Kaiser Permanente, Denver

Department of Family Medicine, University of Colorado Denver, Aurora, CO

Funded by the Agency for Healthcare Research and Quality (1R01 HS018404-01, PI: E.A.B.).

The authors declare no conflict of interest.

Reprints: Chan Zeng, PhD, Institute for Health Research, Kaiser Permanente, 10065 East Harvard Ave, Ste 300, Denver, CO 80231-5968. E-mail:

Article Outline

Morbidity is a potentially important independent predictor or confounder in observational studies evaluating health outcomes.1–6 Administrative datasets, including pharmacy records and diagnosis codes, have become common sources of information regarding morbidity. A variety of summary morbidity measures or scores have been developed on the basis of these data sources.7–11 Studies have shown that higher morbidity scores are associated with poorer outcomes, including increased mortality,3,7,10–12 higher rates of inpatient admissions,10,11,13 and greater risk of long-term care admissions.3

Many of the methods developed to measure morbidity7,8 incorporate diagnoses for chronic and/or episodic conditions into a simple or weighted count of conditions. Typically, these codes are captured for a discrete period of time around an index event, for example 6 months or 1 year. More recently, studies have used longitudinally measured morbidity as a predictor of subsequent health outcomes.14–16 History of comorbid disease—for example slow, steady accumulation of conditions versus a catastrophic event—may influence some outcomes more than a simple count at a point in time. Wang et al14 investigated longitudinal morbidity measurement using the Romano adaptation of the Charlson weighted morbidity score17 to incorporate diagnoses assigned from 1991 to 1999 in a fee-for-service Medicare population. The authors developed 11 different approaches to incorporate morbidity and determined that a combination of baseline morbidity and the prior year’s rolling morbidity performed the best in predicting survival. They recommended further research in different study populations to determine whether longitudinal measures more accurately predict outcomes other than survival.

The trajectory, or pattern, of repeated, longitudinal morbidity measurements before an index event may more accurately predict outcomes following the index event than do cross-sectional morbidity measures. For example, it is plausible that an individual who gradually incurs new comorbid diseases over many years may use healthcare services differently than an individual who rapidly acquires new comorbid conditions. A growth curve model based on a linear mixed effects model is helpful in examining this pattern of change over time, and it can be used to predict subsequent outcomes.18 In addition to trajectories, different starting and ending morbidity values and rates of morbidity change may be associated with differences in outcomes.

The objectives of this study were to: (1) develop statistical models to represent the trajectory of individual morbidity measures over time before an index event or date; (2) compare the performance of morbidity trajectory versus other summary morbidity measures in predicting a range of health outcomes in a population of patients with multiple complex conditions; and (3) confirm our findings in a larger but similar population.

Back to Top | Article Outline


Study Designs and Populations

The setting for this study was Kaiser Permanente Colorado, an integrated, not-for-profit health system. The data for the model development and analysis were collected as part of a prospective cohort study that aims to assess health outcomes as a function of biopsychosocial factors and continuity of care.19 The primary cohort for this study consisted of members of the health plan who were aged 65 years and older, were enrolled in the health plan for at least a year, had 3 or more of 10 common chronic medical conditions, and responded to a survey assessing factors potentially associated with health outcomes. Morbidity (based on ICD-9-CM diagnosis codes) was assessed for up to 10 years before the time that the baseline survey was conducted (2010–2011). Members meeting the same inclusion criteria who were not surveyed comprised the secondary cohort; their morbidity was assessed between November 1999 and October 2009. Age, sex, race/ethnicity, socioeconomic status, and years of enrollment were derived from administrative data sources based on the electronic medical record. We used the primary cohort for model development, using self-reported and administrative health outcomes. To compare the results using the larger cohort, we were restricted to administrative outcomes.

This investigation was approved by the Kaiser Permanente Colorado institutional review board.

Back to Top | Article Outline
Study Outcomes

The primary health outcomes for the initial cohort were clinical measures obtained from administrative sources and self-reported health status. The clinical measures included number of physician office visits, inpatient admissions, emergency department (ED) visits, and mortality during the year after baseline survey (or during the year following October 30, 2009 for the secondary cohort). Because many patients (∼75%) had no inpatient admissions or ED visits during the 1 year follow-up period, we dichotomized these outcomes into none or any. General health status was derived from the well-validated RAND 36 questionnaire.

Back to Top | Article Outline
Measure of Morbidity

The independent variable of interest in our study was morbidity. We used the Quan adaptation of the Charlson Comorbidity Index (CCI),9 which is a diagnosis-based morbidity measure originally derived from medical record review, which has since been adapted for use with the International Classification of Diseases, Ninth Edition (ICD-9) diagnostic codes. The CCI is a weighted score comprising 17 conditions, of which all but 1 (peptic ulcer disease) can be considered chronic. The original Charlson index and its adaptations have demonstrated construct validity in both primary care and hospital populations for predicting multiple outcomes, including mortality, postoperative complications, length of stay, costs of hospitalization and chronic disease care, and admission to skilled nursing facilities.7,9,20 It also has established associations with health-related quality-of-life outcomes, such as self-reported health and functional status.21–24

We calculated morbidity burden for each participant using the following methods:

“Snapshot” CCI was the score in the year before the survey, using only those diagnoses that were coded and captured in that year.

Cumulative CCI was the score over the 10 years before the survey. With the exception of peptic ulcer disease, once a person was diagnosed with a condition, it remained a part of that person’s cumulative score, even if the condition did not appear in administrative data sources in an ensuing year.

Baseline CCI was the CCI calculated for the earliest year of enrollment in the 10-year period. The annual rate of CCI change for each individual was defined as the difference between the last cumulative CCI and first CCI (baseline CCI) divided by the number of years for which a person had CCI measures.

Finally, a CCI trajectory was modeled using mixed effect regression models (or growth curve models) to fit each individual’s CCI measures using available data for up to 10 years. We modeled both linear and quadratic effects of time, as well as adjusting for individual demographic characteristics, to determine the best fit for the data. The estimated intercept and slope parameters describing individual CCI trajectories were used as independent variables in the subsequent outcome model.

Back to Top | Article Outline
Statistical Analysis

We modeled health outcomes as a function of the morbidity measures described above. We used a linear regression model for the continuous outcome of health status. For outcomes in the year after the survey, we used a negative binomial regression model for counts of primary care visits and logistic regression models for dichotomized outcomes of ED visits, inpatient admissions, and death. Age, sex, race/ethnicity, years of health plan enrollment, and socioeconomic status (based on census data) were considered as potential covariates; the interactions between morbidity measures and covariates were tested for significance as well.

We used the Akaike information criterion (AIC)25 to compare the relative goodness of fit of the models using the different morbidity measures. The AIC provides an objective way of determining which model among a set of models is most parsimonious. The model with smallest AIC is considered to be the best model. To quantify the AICs of each model relative to the best model, we calculated the delta AIC (Δi), which is defined as: Δi=AICi–AICmin, where AICi is the AIC value for model i, and AICmin is the AIC value of the best model. As recommended by Burnham and Anderson,26,27 models with Δi values <2 are considered essentially as good as the best model, and models with Δi values >10 are sufficiently poorer than the best model to be considered implausible

AIC is useful in identifying best goodness of fit, but this does not necessarily translate into good predictive or discriminative ability. Although we were not using morbidity indices to build full predictive models, we assessed the relative strength of each morbidity measure using an R squared (R2) for linear regression models for continuous outcomes, a modified R2 for Poisson or negative binomial regression models for count outcomes,28–31 and the C-statistic for logistic regression models. All analyses were conducted using SAS version 9.2 (SAS Institute, Cary, NC)

Back to Top | Article Outline


A total of 961 members who participated in the original telephone survey comprised the primary cohort for the model development. Seventy percent (n=669) had 10 years of membership before the survey, and only 76 (2.7%) had <5 years of membership. Fifty-five percent were female, and average age at time of survey was 75.6 (SD 5.7). Table 1 displays the demographic characteristics and survey health outcomes collected at the time of survey and clinical outcomes within 12 months after survey for our population. Figure 1 presents the trajectory of cumulative CCI scores for 5 selected subjects from the primary cohort; they all had the same CCI score in the final year, but their trajectories were very different.

Our secondary cohort consisted of 13,163 patients. Their inclusion criteria were identical to those of the primary cohort; 1113 (8%) were randomly selected to be contacted for the survey but were not surveyed for various reasons (mostly refused or were unable to contact); the remaining 92% were not sampled. Their clinical characteristics were very similar to those of our primary cohort, but they had a higher proportion of people with low socioeconomic status (17.3%), a shorter length of enrollment (average 7.6 y), and a higher mortality rate (4.7%) in the 1 year follow-up after the survey.

When estimating the CCI trajectories, we found that models adjusted for covariates were not superior to those without and that the model with both linear and quadratic time fit the data statistically better than the model that included only linear time (P<0.001). Table 2 contains the description and distributions of the morbidity measures for the primary cohort. The observed baseline CCI ranged from 0 to 9 (median=0), the snapshot CCI ranged from 0 to 12 (median=1), and the cumulative CCI ranged from 0 to 13 (median=3). The estimated linear slope from the mixed effect linear model was similar to the observed rate of CCI change. Although the positive sign of the quadratic slope suggested that the CCI trend accelerated over time, the actual effect was very modest and not visually distinct from the linear model (Fig. 2). Therefore, we evaluated the variables representing an individual’s morbidity trajectory from both models: intercept (estimated baseline CCI at 10 y before survey) and linear slope (estimated rate of CCI change) from the linear model; and intercept, linear slope, and quadratic slope from the quadratic model.

Covariates and interactions between CCI measures and covariates that were not significantly associated with our outcomes were excluded from models. The results in the form of coefficients, standard errors, AIC, and predictive or discriminative ability for the primary cohort are presented in Table 3 for all of the health outcomes as a function of the CCI variations. The models using the estimated intercept and slope from the quadratic and linear models had the best fit and highest R2 for self-reported general health status, but were only marginally better than the other CCI measures. Although all the CCI measures showed a significant association with self-reported general health score—that is, a higher CCI score at baseline or a year before survey, or more rapid change of CCI from baseline associated with lower (worse) general health—they only accounted for 7%–8% of the variability in the health status outcome.

For the outcome of mortality in the year after the survey, the snapshot CCI yielded the lowest AIC (best model) and highest C-statistic (greatest discrimination). C-statistics for all models ranged from about 0.80 to 0.84, indicating excellent discriminative ability.

For utilization outcomes in the year after the survey, cumulative CCI provided the best model fit for ED and office visits. Although the snapshot CCI provided the best model fit for inpatient admission, the cumulative measure and observed pattern of change had comparable results. The observed pattern of change and the linear model had only slightly higher AICs in modeling ED visits.

The small R2 for health status and modified R2 for office visits indicated that, regardless of CCI measure used, morbidity explained very little of the proportion of variability in the outcomes. C-statistics for modeling inpatient admission and ED visit were in the range of 0.60–0.63, indicating poor discriminative ability.

Using the secondary cohort, our results in terms of mortality and utilization outcomes were similar to those among the primary cohort: models using the snapshot or cumulative CCI had better model fit than those using other CCI measures (Table 3).

Back to Top | Article Outline


Morbidity burden is an important predictor of a broad array of health outcomes. As that burden tends to increase over time, there have been concerns that measuring morbidity at a single time point may lead to incorrect or simplistic conclusions about the relationship between morbidity and specific outcomes. To address this concern, we compared a standard approach to measuring morbidity with models that incorporated the acquisition of chronic conditions at differing rates over time, and evaluated the relationship between those morbidity measures and an array of objective (utilization, mortality) and subjective (self-report) health outcomes. We used a growth curve model based on individual longitudinal measures of CCI to examine overall patterns of change in morbidity over time. The estimated intercept and slope parameters of CCI were then included in models as predictors of subsequent health outcomes. We then compared models with these estimated CCI trajectories with models using observed baseline CCI and rate of CCI change, a cumulative 10-year compilation of the CCI, and a traditional 1-year calculation of the CCI.

Our findings suggest that using multiple years of diagnoses for chronic conditions (ie, not a “snapshot”) may slightly improve the fit and predictive ability of models looking at a range of health outcomes: ED and office visit utilization and self-reported health status. Incorporating the pattern of change in morbidity over time, however, proved superior to other morbidity measures only in modeling health status. In contrast, using a standard, 1-year “snapshot” of morbidity performed best in modeling death and inpatient admissions in our primary cohort, as well as all the outcomes in the secondary cohort. It may be that death and hospitalization, in particular, have a stronger association with more proximal morbidity, regardless of history or pattern of morbidity. Although we did see some differences in results between the primary and secondary cohorts in terms of ED and office visits, in neither case was morbidity trajectory the best model.

For some outcomes, the difference between the “best” AIC model and other models was minimal. This was the case for health status (estimated CCI trajectories); likelihood of an inpatient admission (snapshot and cumulative CCIs); likelihood of an ED visit (cumulative CCI, observed CCI baseline and rate of change, and estimated CCI baseline CCI and linear rate of change); and number of ED visits (cumulative CCI and estimated CCI baseline CCI and linear rate of change). With the exception of likelihood of inpatient admission, this would seem to indicate that the method used to calculate the CCI does not substantially affect its association with service utilization or mortality.

Although using a standard, 1-year “snapshot” of morbidity was sufficient to predict some of the outcomes in our study, the fact that the mean cumulative CCI in our population was 73% higher than the mean snapshot CCI shows that chronic conditions are not consistently documented over time and that using longer time frames to capture diagnoses improves that capture. Thus, for a more accurate representation of morbidity, longer time frames should be used if possible.

Our results for the outcome of mortality are not consistent with those of Wang et al,14 who found that a combination of baseline comorbidity and the last year’s rolling comorbidity was best in modeling time to death in a seemingly similar population in terms of age, sex, and race. We also examined models of time to death for our primary cohort, but the results were consistent with modeling death as a dichotomized outcome (data not shown). One possible explanation is that our population was enrolled in an integrated health plan, and had relatively long periods of membership (70% of our primary cohort with 10 y or more and <3% with <5 y). In the Wang study, individuals were enrolled in a fee-for-service Medicare plan, and nearly 10% of individuals contributed only 1 year of morbidity data; when the cohort was restricted to individuals with at least 2 years of enrollment, the model suggested that using only the prior year rolling comorbidity variable was sufficient.

In addition to Wang,14 a few other investigators have explored the use of longitudinal morbidity. Among a population of community-based adults aged 51 and older, Quinones et al32 used annual self-report of 7 diseases, in addition to health status, healthcare utilization, and other time-varying factors, to create linear trajectories of comorbidity. They assessed racial and ethnic variations and found differences in the trajectory of disease between Black, White, and Mexican Americans, but they did not assess the relationship between comorbidity and outcomes. Zimmer et al,15 estimated disability and mortality trajectories over time in a population aged 80 and older in China, but their focus was on predictors of trajectory group membership, rather than outcomes.

Even though our population consisted of health plan members aged 65 and older with multiple health conditions, many of the adverse outcomes we assessed, that is, death or ED or inpatient utilization, were uncommon. Most individuals reported fairly good health status. They tended to have long periods of enrollment in the health plan. Although we confirmed our findings in a larger secondary cohort, our results may still not be generalizable to populations that do not have consistent access to integrated healthcare services, who have some predefined condition or procedure that puts them at greater risk for worse outcomes, or who are younger and less likely to have multiple chronic conditions. In addition, we were not able to compare findings across the 2 cohorts on the important outcome of health status. Finally, our conclusions are limited to morbidity measurement based on the CCI measure; other morbidity measures (and/or different modeling strategies such as nonparametric models) may provide different longitudinal trajectories for morbidity burden. In addition, morbidity measures by themselves account for little variability or discriminative ability relative to outcomes; our results were consistent with other studies that have reported R2 and C-statistics.3

To our knowledge, this is the first study to apply growth curve analyses to longitudinal measures of comorbidity to evaluate associations with a variety of objective and self-report health outcomes. We expected patterns of morbidity change over time to add to the predictive ability of models looking at these associations, but we did not see overwhelming evidence to support this. Thus, we conclude that total morbidity burden may be as important as the rate of accumulation of morbidities in predicting these outcomes.

Back to Top | Article Outline


1. Klabunde CN, Warren JL, Legler JM.Assessing comorbidity using claims data: an overview.Med Care.2002;40:IV-26–35.
2. Schneeweiss S, Maclure M.Use of comorbidity scores for control of confounding in studies using administrative databases.Int J Epidemiol.2000;29:891–898.
3. Schneeweiss S, Seeger JD, Maclure M, et al..Performance of comorbidity scores to control for confounding in epidemiologic studies using claims data.Am J Epidemiol.2001;154:854–864.
4. Chu YT, Ng YY, Wu SC.Comparison of different comorbidity measures for use with administrative data in predicting short-and long-term mortality.BMC Health Serv Res.2010;10:140.
5. Farley JF, Harley CR, Devine JW.A comparison of comorbidity measurements to predict healthcare expenditures.Am J Manag Care.2006;12:110–119.
6. Tang J, Wan JY, Bailey JE.Performance of comorbidity measures to predict stroke and death in a community-dwelling, hypertensive Medicaid population.Stroke.2008;39:1938–1944.
7. Charlson ME, Charlson RE, Peterson JC, et al..The Charlson comorbidity index is adapted to predict costs of chronic disease in primary care patients.J Clin Epidemiol.2008;61:1234–1240.
8. Elixhauser A, Steiner C.Harris DRet al. Comorbidity measures for use with administrative data.Med Care.1998;36:8–27.
9. Quan H, Sundararajan V, Halfon P, et al..Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data.Med Care.20051130–1139.
10. Von Korff M, Wagner EH, Saunders K.A chronic disease score from automated pharmacy data.J Clin Epidemiol.1992;45:197–203.
11. Clark DO, Von Korff M, Saunders K, et al..A chronic disease score with empirically derived weights.Med Care.1995;33:783–795.
12. Schneeweiss S, Wang PS, Avorn J, et al..Improved comorbidity adjustment for predicting mortality in Medicare populations.Health Serv Res.2003;38:1103–1120.
13. Putnam SL.Alcoholism, morbidity and care-seeking: the inpatient and ambulatory service utilization and associated illness experience of alcoholics and matched controls in a health maintenance organization.Med Care.1982;20:97–121.
14. Wang CY, Baldwin LM, Saver BG, et al..The contribution of longitudinal comorbidity measurements to survival analysis.Med Care.2009;47:813–821.
15. Zimmer Z, Martin LG, Nagin DS, et al..Modeling disability trajectories and mortality of the oldest-old in China.Demography.2012;49:1–24.
16. Morris RL, Sanders C, Kennedy AP, et al..Shifting priorities in multimorbidity: a longitudinal qualitative study of patient’s prioritization of multiple conditions.Chronic Illn.2011;7:147–161.
17. Romano PS, Roos LL, Jollis JG.Adapting a clinical comorbidity index for use with ICD-9-CM administrative data: differing perspectives.J Clin Epidemiol.1993;46:1075–1079.
18. Maruyama N, Takahashi F, Takeuchi M.Prediction of an outcome using trajectories estimated from a linear mixed model.J Biopharm Stat.2009;19:779–790.
19. Bayliss EA, Ellis JL, Shoup JA, et al..Association of patient-centered outcomes with patient-reported and ICD-9-based morbidity measures.Ann Fam Med.2012;10:126–133.
20. Deyo RA, Cherkin DC, Ciol MA.Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases.J Clin Epidemiol.1992;45:613–619.
21. Andresen EM, Bowley N, Rothenberg BM, et al..Test-retest performance of a mailed version of the Medical Outcomes Study 36-Item Short-Form Health Survey among older adults.Med Care.1996;34:1165–1170.
22. Bari M, Virgillo A, Matteuzzi D, et al..Predictive validity of measures of comorbidity in older community dwellers: the Insufficienza Cardiaca negli Anziani Residenti a Dicomano Study.J Am Geriatr Soc.2005;54:210–216.
23. Heller DA, Ahern FM, Pringle KE, et al..Among older adults, the responsiveness of self-rated health to changes in Charlson comorbidity was moderated by age and baseline comorbidity.J Clin Epidemiol.2009;62:177–187.
24. Salaffi F, Carotti M, Stancati A, et al..Health-related quality of life in older adults with symptomatic hip and knee osteoarthritis: a comparison with matched healthy controls.Aging Clin Exp Res.2005;17:255–263.
25. Akaike H.A new look at the statistical model identification.IEEE Trans Automatic Control.1974;19:716–723.
26. Burnham KP, Anderson DR.Model selection and multi-model inference: a practical information-theoretic approach.2002.New York:Springer.
27. Burnham KP, Anderson DR.Multimodel inference understanding AIC and BIC in model selection.Sociol Methods Res.2004;33:261–304.
28. Cameron AC, Windmeijer FAG.R-squared measures for count data regression models with applications to health-care utilization.J Bus Econ Stat.1996;14:209–220.
29. Heinzl H, Mittlbock M.Pseudo R-squared measures for Poisson regression models with over-or underdispersion.Comput Stat Data Anal.2003;44:253–271.
30. Mittlbock M, Waldhor T.Adjustments for R2 measures for Poisson regression models.Comput Stat Data Anal.2000;34:461–472.
31. Mittlbock M.Calculating adjusted R2 measures for Poisson regression models.Comput Methods Programs Biomed.2002;68:205–214.
32. Quinones A, Liang J, Bennett JM, et al..How does the trajectory of multimorbidity vary across Black, White, and Mexican Americans in middle and old age?J Gerontol B Psychol Sci Soc Sci.2011;66:739–749.

morbidity; trajectory; pattern; Charlson Comorbidity Index (CCI); prediction

© 2014 by Lippincott Williams & Wilkins.