In this issue of EPIDEMIOLOGY, Goldstein and colleagues^{1} present “a new statistical method for estimating influenza-related mortality” using recent US mortality and influenza data sources. They propose that their method is an advance over previous methods^{2–13} for estimating influenza disease burden. Strengths of their approach include: (1) the accommodation of nonlinear mortality effects of specific influenza types and subtypes over time; (2) the use of a logical method to model exposure to influenza; and (3) an assessment of autocorrelation in the time series. There are also some limitations, which include (1) questionable statistical assumptions in the use of linear regression models for weekly data with low frequency counts; (2) neglect of recent advances in time-series modeling; (3) a complex and nonparsimonious model; and (4) lack of comparisons of influenza-associated mortality estimates with those from the published literature using similar outcomes.

#### A BRIEF HISTORY

Indirect statistical methods have long been used to estimate the number of deaths associated with influenza, often making use of vital statistics. Estimates of influenza-associated deaths were first made in 1847 in England by William Farr14 and were subsequently made for the 1892 Massachusetts pandemic and 1918 pandemic15—decades before actual human influenza viruses were isolated.16 For many years, investigators relied on indirect methods because they lacked inexpensive, sensitive, and widely available diagnostic tests for influenza infection. More recently, reverse transcription polymerase chain reaction (RT-PCR) assays provide a practical gold standard for detecting infections with influenza and other respiratory virus infections.^{17},^{18} Recent studies have estimated influenza-associated hospitalizations on a population-level through prospective enrollment of hospitalized subjects followed by RT-PCR testing of respiratory specimens.^{19},^{20} Still, the availability of molecular diagnostics has not made time-series methods obsolete. Influenza-associated deaths will always be difficult to count. Older adults, who are at greatest risk of influenza- associated mortality, may not die from the initial assault of the virus but from later bacterial superinfection, or from exacerbation of chronic respiratory or cardiac conditions. Such deaths can occur a week or two after the acute infection, by which time the virus is no longer replicating.^{21},^{22}

#### GENERAL MODEL STRATEGIES

There are three broad classes of models to estimate deaths associated with infectious agents. The etiologic fraction method takes the proportion of specific infection among proximal outcomes and applies this to an outcome for which it is more difficult to obtain etiologic data, such as mortality.^{23} (For example, the proportion of all pneumonia deaths due to pneumococcus can be estimated using the proportion of all pneumonia hospitalizations due to pneumococcus.)^{24} The obvious question is whether the agent of interest infects similar proportions of those hospitalized and of those dying. For influenza, this remains unknown.

A second class of models is the probabilistic multiplier approach,^{25} which estimates the mortality rate as the product of the symptomatic attack rate and the risk of death given infection (case-fatality rate).^{26} Data of this type are seldom available, limiting this model’s main use to influenza pandemics or other epidemics that attract the needed resources to estimate the needed parameters.

The third class of models, and the one most frequently used in influenza studies, is time-series regression. The model proposed by Goldstein and colleagues^{1} is an example. Time-series models have been used for at least five decades to estimate influenza-related deaths in the United States and other temperate countries with relatively consistent, well-defined periods of influenza circulation during winter months.^{11},^{27–29} We would suggest that the time-series approach of Goldstein and colleagues does not take into account some of the advances in time-series modeling as applied to infectious diseases.

The best-known of the time-series models is the original Serfling regression model.^{30} This model includes cyclical regression terms (sine and cosine terms) to account for the obvious seasonality of mortality in the United States. Both the initial model and subsequent iterations predict that deaths exceeding a seasonally adjusted baseline are all influenza-associated, given that most if not all of the excess deaths occurred during the same time period that influenza viruses were circulating. Serfling proposed to do this by setting death counts in particular weeks determined to be significant outliers to an estimated baseline constructed after omitting those data points from the original time series. Variations in the original Serfling regression model have been driven largely by the type and quality of data available. Time-series methods have been used to estimate influenza burden for a wide range of health outcomes, including outpatient visits,^{31},^{32} hospitalizations,^{4},^{7},^{31},^{32} and mortality.^{2},^{3},^{8},^{30},^{33},^{34}

One of the most important advances in modeling influenza-associated deaths was made by Clifford and colleagues.^{35},^{36} They proposed an alternative to the Serfling model; in the Clifford model, laboratory-diagnosed virus infections and influenza diagnosis rates were used in a stepwise regression analysis that also included terms for secular trends, seasonal patterns, air temperature, and number of years since a major change was detected in the influenza virus hemagglutinin antigen. The Clifford model was novel in that it did not impute the mortality baseline. Rather, it estimated excess deaths by subtracting the predicted baseline values (with the influenza terms set to zero) from the estimated values during the same time period when influenza viruses were circulating. The Clifford model was subsequently adopted by Alling and colleagues^{37},^{38} and used to estimate excess influenza deaths in the United States.

During the late 1990s, weekly US data became available for the numbers of respiratory specimens tested, and the numbers of samples that tested positive for influenza and respiratory syncytial virus (RSV). These data provided a new opportunity to model influenza-associated deaths for both influenza and RSV simultaneously, using methods that directly account for the temporal identification of virus activity. Several research groups began to explore such models. For example, quasi-Poisson regression models were applied to US viral surveillance data for influenza A(H1N1), B, A(H3N2), and RSV viruses to estimate burden of pneumonia and influenza deaths, respiratory and circulatory deaths, and all-cause mortality.^{8} This general methodology was subsequently applied to hospitalizations and other health outcomes^{7},^{39} and compared with other burden of disease models and estimates.^{9} Time-series methods have also provided an opportunity to add other important covariates and confounders into the models, including time-varying exposure models for air pollution effects on mortality.^{40},^{41},^{42} These models are increasingly used in regions with more complex seasonal influenza patterns.^{27},^{43} Based on this biostatistics and influenza literature, it appears that Goldstein and colleagues^{1} might have missed an opportunity to provide stepping stones to the next iteration of influenza burden of disease methods. It is difficult to evaluate how likely violations of statistical assumptions might have affected their results. This assessment is complicated by the use of different outcome measures in Goldstein’s models relative to the previous literature—thus precluding direct historical comparisons of results from other studies.

#### MODELING LOW-FREQUENCY-COUNT DATA

Our second concern with the approach proposed by Goldstein et al^{1} is that it does not capitalize on the advances in time-series methods for low-frequency-count data. Their approach instead uses linear regression models for outcomes likely to have very low frequencies. An important advance in generalized linear models in the early 1970s^{44} allows for more complicated forms of regression modeling that relax the assumption of linearity between predictor and response variables. Generalized linear modeling permits the expected value of the response variable to be a smoothed (eg, nonlinear) monotonic function of the linear predictors. Generalized linear modeling also relaxes the assumption of normal distribution for the response variable, allowing any distribution from the exponential family. For count data, means and variances can be modeled by discrete probability distributions, the most common being the Poisson, binomial, negative binomial, and geometric distributions. Poisson regression is a generalized linear model with Poisson-distributed response and a natural logarithm for the link function. This has become a common choice for epidemiologic models because it is well suited to low-frequency-count data like mortality.^{40},^{41,45,46} Although the choice of the link function used in generalized linear modeling is somewhat arbitrary, the function should nonetheless support the same range of values as the probability distribution (eg, a Poisson distribution is supported on [0, ∞], so a good link function would have a domain of [0, ∞], such as the natural logarithm function).

Poisson distributions are themselves limited by having a mean equal to the variance. This assumption is frequently violated when working with low-frequency-count data, and thus quasi-Poisson or negative binomial regression techniques are widely used in time-series studies using death certificate data.^{43},^{47–49} In these models, a second parameter is used to account for either overdispersion (variance > mean) or underdispersion (variance < mean). Other models—including zero-inflated,^{50} hurdle,^{51} finite mixture,^{52} generalized estimating equations,^{45} mixed/hierarchical generalized linear models,^{53} and generalized additive models^{54}—can also provide flexible variance structures when working with mortality data. When modeling mortality data from national vital records, quasi-Poisson or negative binomial estimation techniques have statistical advantages (eg, fewer assumptions required) as well as practical advantages (interpretability) over standard linear regression. The limitations of traditional linear regression in estimating count-derived data have been noted in many fields.^{55–60} A serious limitation in the approach of Goldstein et al^{1} is their assumption that mortality count data can be adequately modeled with a normal probability distribution and corresponding error terms.

#### MODEL COMPLEXITY

Our third concern is with regard to parsimony. Goldstein et al^{1} have chosen to fit a relatively complex model that has many underlying statistical and biological assumptions. The authors apply a very specific model (eg, modeling changes in the effects of influenza A(H3N2) across a very short time period. This is likely to be too short a period for modeling such differences and is unlikely to be generalizable to other modeling efforts. Goldstein and colleagues also choose to include an autocorrelation term between subsequent observations. This is a common technique in many time-series applications (e.g. market analysis), but based on our own experience, the addition of autocorrelated terms reduces estimates substantially because the model absorbs some of the variance likely associated with influenza. Goldstein and colleagues suggest that, even though their model is different (and more complex), it results in estimates similar to those based on less complex models. If parsimony is a virtue, one might then prefer instead to select the less complex model with fewer assumptions, at least until arguments of biological plausibility supporting such complexity could be developed from other data.

#### MODEL VALIDITY

Our last concern is with the suggestion that the proposed methodology is an advance over previous approaches. It is a truism that “all models are wrong; some are useful.”^{61} Most burden of disease estimates have been consistent regardless of statistical method, showing substantial season-to-season variations in influenza-associated deaths.^{9} Given that evidence of recent influenza infection is undetectable in most persons dying during peak influenza circulation, there is no gold standard for determining a “best model.” It is therefore important for researchers proposing new models to make direct comparisons of their results with those in the literature.

In sum, we believe most epidemiologists would agree on certain guidelines in model development: (1) use a model type suitable for the distribution and error structure of the data; and (2) invest resources in validation and cross-validation techniques that can help verify whether indirect modeling is both internally and externally consistent. The full contribution of the model proposed by Goldstein and colleagues is difficult to assess before such information is in hand.

#### ABOUT THE AUTHORS

*WILLIAM THOMPSON is the Associate Director of Science for the Division of Behavioral Surveillance in the Office of Surveillance, Epidemiology and Laboratory Services, CDC. His interests include influenza modeling and behavioral risk factor research. BENJAMIN RIDENHOUR is an Assistant Professor in the Department of Biological Sciences at Notre Dame. His interests include biostatistical and biomathematical modeling of disease systems. JOHN BARILE is an Assistant Professor at the University of Hawaii at Mānoa. His interests include the study of environmental contributors of health. DAVID SHAY is a medical epidemiologist with CDC’s Influenza Division. His interests include burden of disease models for viral respiratory pathogens and vaccine effectiveness studies.*