Secondary Logo

Journal Logo


Statistical Power for Trend-in-trend Design

Ertefaie, Ashkan; Small, Dylan S.; Ji, Xinyao; Leonard, Charles; Hennessy, Sean

Author Information
doi: 10.1097/EDE.0000000000000803

To the Editor:

Unmeasured confounding is often a major concern in observational studies. The recent article by Ji et al.1 proposed a novel design known as trend-in-trend that, under certain assumptions including a strong time trend in exposure prevalence, provides unbiased estimates of the effects of exposures in the presence of unmeasured confounding. It accomplishes this by examining trends in outcome occurrence as a function of trends in exposure prevalence across strata defined by the cumulative probability of exposure, which models exposure as function of measured baseline variables and effectively stratifies on rate of adoption of the exposure. It therefore extends and improves on studies using calendar time as an instrumental variable,2–4 eliminating its reliance on the assumption of an absence of a secular trend in the outcome.

Several factors affect the statistical power/detectable alternative of trend-in-trend studies, although we know of no closed-form solution to their estimation. We therefore developed a Monte Carlo simulation approach for estimating statistical power or detectable alternative when planning a trend-in-trend study. This approach requires the investigator to specify six parameters: (1) the type-1 error rate; (2) the probability of a study subject experiencing the study outcome during any study interval; (3) the c statistic of the cumulative probability of exposure model5; (4) the number of cumulative probability of exposure strata into which the population is divided; (5) the shape of the exposure trend, expressed as a linear or quadratic function of time on log scale; and (6) the desired statistical power or minimum detectable causal odds ratio. The simulation procedure (which has been incorporated into the TrendInTrend package for the R: provides an estimate for either the statistical power or the minimum detectable odds ratio, whichever was specified in (6) above. The eAppendix ( provides technical information about the simulation procedure.

To illustrate the simulations and assess the influence of the required parameters, we estimated the statistical power of a hypothetical trend-in-trend study under different scenarios. We assumed (1) a type-1 error rate of 5%; (2) a proportion of the population experiencing the outcome during any study interval of 0.007, 0.018, and 0.049; (3) a c statistic for the cumulative probability of exposure model of 0.50, 0.60, and 0.75; (4) a number of strata of 5 and 10; (5) a linear time trend in exposure prevalence with slopes (αt) 0.07 and 0.20 relative percent over the study period (Figure A). We assumed that the study period was divided into 10 time intervals, and generated 500 datasets of size 10,000 (1,000 of whom were ever-exposed) for each scenario.

The trend-in-trend design power analyses. The dashed and dotted lines represent a weak and a strong exposure trend. A, The exposure prevalence over time. B, Statistical power for odds ratios of 1–2 for three different values of c statistic of the cumulative probability of exposure model: 0.75 (green), 0.60 (red), and 0.50 (black). The outcome rate is set to be 0.01. C, Statistical power for odds ratios of 1–2 where outcome probability is 0.049 (green), 0.018 (red), and 0.007 (black), where the c statistic is set to 0.75. D, Statistical power for odds ratios of 1–2 for number of cumulative probability of exposure strata = 5 (black) or 10 (red). In all scenarios, the type I error rate is 0.05, the number of intervals is 10, the number of individuals is 10,000 and the number of replications is 500.

The Figure B–D displays simulated statistical power over odds ratios ranging from 1 to 2, with the dashed and the dotted lines representing weak

and strong

time trends in exposure, respectively. The Figure B shows the effect of the c statistic of the cumulative probability of exposure model and strength of exposure trend on power when the outcome rate over the entire study period is set to 0.015 and the number of cumulative probability strata is set to 5. The strength of exposure trend has a bigger influence on power than does the c statistic, although c statistic does have an observable effect. The Figure C shows the influence of the outcome probability when the c statistic is set at 0.75 and the number of cumulative probability of exposure strata is set to 5. As the outcome probability increases, power increases dramatically. The Figure D assesses the influence of number of cumulative probability of exposure strata, setting the c statistic to 0.75 and outcome probability to 0.015. It shows that when there is a strong exposure trend (dotted lines), the power function is not sensitive to the number of strata, while for the weak trend (dashed line), increasing the number of strata from 5 (black) to 10 (red) actually reduces the power. Comparing the Figure B–D suggests that the influence of the outcome probability is stronger than that of the c statistic or number of cumulative probability of exposure strata. Nonetheless, our results show that when there is a strong exposure prevalence trend, the trend-in-trend design has a reasonable statistical power regardless of outcome rates, c statistic values, and number of strata.

Ashkan Ertefaie

Department of Biostatistics and Computational Biology

University of Rochester

[email protected]

Dylan S. Small

Xinyao Ji

Department of Statistics

University of Pennsylvania

Philadelphia, PA

Charles Leonard

Sean Hennessy

Center for Pharmacoepidemiology Research and Training

Center for Clinical Epidemiology and Biostatistics

Perelman School of Medicine at the University of Pennsylvania

Philadelphia, PA


1. Ji X, Small DS, Leonard CE, Hennessy SThe trend-in-trend research design for causal inference. Epidemiology. 2017;28:529–536.
2. Angrist J D, Imbens G W, Rubin D BIdentification of causal effects using instrumental variables. J Am Stat Assoc. 1996;91:444–455.
3. Hernán MA, Robins JMInstruments for causal inference: an epidemiologist’s dream? Epidemiology. 2006;17:360–372.
4. Ertefaie A, Small DS, Flory JH, Hennessy SA tutorial on the use of instrumental variables in pharmacoepidemiology. Pharmacoepidemiol Drug Saf. 2017;26:357–367.
5. Steyerberg EW, Harrell FE Jr, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JDInternal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54:774–781.

    Supplemental Digital Content

    Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved.