Data-driven optimization of health system efficiency is critical to sustainably improving service delivery. In low-resource health systems, data-generating structures are frequently built to report specific indicators “up”—to health authorities and donors. This often comes at the expense of frontline health care workers (HCWs) using routine data to guide contextually appropriate strategies to improve service delivery.1 Engaging HCWs to use routine program data to inform changes can make this time-intensive activity worthwhile to HCWs and improve data quality. Routine program data have great potential to inform data-driven optimization, however, with potential quality limitations.2,3
Cascade analysis is an implementation strategy that uses data to visualize and quantify system performance, inefficiencies, and bottlenecks. Cascades are characterized by a series of sequential, conditional steps in a process that yield a desired outcome and explicitly model “handoffs” between and across complex, interdependent system components. For example, the prevention of mother-to-child HIV transmission (PMTCT) cascade involves HIV testing, linkage to care, antiretroviral therapy (ART) for HIV-positive mothers, antiretroviral (ARV) prophylaxis for infants, and repeated infant testing. Cascade analysis has informed numerous disease models and goal setting (eg, UNAIDS 90-90-90 goals4), identified gaps in system performance (eg, PMTCT5,6 and HIV prevention cascades7,8), and helped prioritization of areas where improvement is most needed. Cascade analysis facilitates thinking across the entire care system to determine which steps are the most inefficient and where to focus improvement efforts to maximize impact.
Despite effectively identifying system inefficiencies, cascade analysis does not improve system performance unless paired with effective improvement strategies. The Systems Analysis and Improvement Approach (SAIA) is a package of implementation strategies that pairs a novel cascade analysis tool (CAT) with established implementation strategies for improvement—process mapping and continuous quality improvement (CQI)—to support frontline HCWs and managers who lead efforts to diagnose inefficiencies and guide iterative improvements in complex delivery systems.9 In a three-country cluster randomized trial, SAIA reduced drop-offs in the PMTCT cascade.10 CAT saved HCW time, enabled prompt feedback for self-assessment and motivation, increased program ownership, and guided managers to equitably allocate resources (including time).
In this article, we describe CAT adaptations for use within HIV and beyond. We define CAT and its components; identify processes, systems, and data sources well and poorly suited to CAT; describe 7 adaptations of CAT; and identify gaps and opportunities in CAT adaptation and implementation for systems improvement.
CAT Structure and Mathematical Logic
CAT is intended to characterize present performance of a system and then aid in the relative prioritization of improvement opportunities. CAT visualizes cascades as a series of sequential, conditional, binary steps. It is a basic, Excel-based simulation model with an optimization function, using inputs from routine program data to identify the cascade step with the maximum opportunity for improvement. A cascade “step” may be either service receipt (e.g., HIV testing) or behavior (e.g., adherence).
We developed the first CAT in 2004 to model the HIV treatment cascade in Mozambique.11 This CAT highlighted discrete cascade steps—(1) HIV testing, (2) ART clinic enrollment, (3) CD4 testing, (4) ART initiation, and (5) ART adherence—to prioritize time and resource allocation. CAT quantifies the number of people lost at each step (those eligible for but not completing each step) and the additional number of patients who would complete all 5 steps if an individual step achieved perfect performance while the performance of all other steps remained constant—what we call “cascade gain.”11
This early CAT was adapted to create the “PCAT” (PMTCT cascade analysis tool)9,12 used in the original SAIA trial. The PCAT presented aggregated data from cross-sectional, month-long periods abstracted from existing registers showing mother–infant dyads. It did not follow specific woman–infant dyads through each step, but rather reflected dyads who completed each step relative to the number who were expected to, based on upstream steps during the same period. The PCAT is more complex than the initial CAT, as PMTCT includes 2 cascades—antenatal care (ANC) initiation through newborn ARV prophylaxis (cascade 1) and enrollment of HIV-exposed infants in care through ART initiation for HIV-infected infants (cascade 2) (Fig. 1). Shaded cells represent data that are entered from PMTCT registers or demographic birth estimates, while unshaded cells represent automatic calculations. For each cascade step, the PCAT shows the number (column A) and proportion (column B) completing that step, number not completing (column C), and the potential “cascade gain” if a given step were to perform optimally (column D).
Cascade gain enumerates dyads that would successfully complete the full cascade if a given step performed perfectly, absent any changes to the remaining steps. In the example in Figure 1, if all mothers in ANC received HIV testing (improvement from 70% to 100%), without downstream improvements, we would expect another 127 HIV-exposed infants to successfully start HIV prophylaxis. The cascade gain column is used for relative prioritization when there are multiple opportunities for system improvement and insufficient resources to address them all simultaneously. For example, 30% gaps in infant linkage to care (step 5) and infant testing (step 6) could intuitively be seen as equally important; additionally, infant testing appears more proximal to the desired outcome than linkage. However, the cascade gain for infant linkage to care is greater; thus, HCWs should prioritize this step for optimal systems improvement in settings with limited time and resources.
Although the calculations are relatively simple, cascade gain—a series of conditional probabilities multiplied by one another and applied to the absolute number of individuals who did not complete the step—is the most novel and challenging concept within CAT. Maintaining the example from Figure 1, 670 women missed HIV testing; if this step were fixed, those 670 women would have been tested, 214 (670*32%) would be expected to have been positive, 193 (670*32%*90%) would be expected to have linked to PMTCT, and 127 (670*32%*90%*67%) would be expected to have brought their infants for HIV prophylaxis, reflecting the “cascade gain” column. This structure assumes counterfactual exchangeability, that is, we assume that if those dyads that failed a step had not failed, they would have had the same subsequent cascade probabilities as those who were observed to have completed the step. Although subsequent CAT adaptations have moderately more complex structures, all have the same logic of sequential multiplied conditional probabilities.
Systems to Which CAT is Well and Poorly Suited
CAT is well suited to care models where (1) steps occur sequentially, (2) the system's goal is to optimize (reach 100% completion) on at least 1 step, and (3) each step is binary (completed or not). There are, of course, conditions and exceptions:
- Although health systems are complex and not every patient follows the same flow, CAT is well suited to systems with a dominant service delivery flow pattern, even if minority flow patterns exist. For example, the PMTCT system has 1 dominant service delivery flow pattern making CAT suitable, while emergency department settings have multiple conditional, branching steps in patient management and highly individualized flow patterns challenging for CAT.
- Some cascade steps are not intended to be optimized. For example, we would aspire to have 100% of pregnant women tested for HIV, but would not aspire to have 100% test HIV-positive. CAT remains well suited to systems with these types of steps, but may be less well suited to a system in which management is less algorithmic and clinical decision-making is based on multiple, individualized factors (comorbidity, clinical presentation, and patient resources).
- Some steps have a number of preferred outcomes based on test value ranges. A continuous variable may be transformed into discrete categorical options; with sufficiently few options, this system may remain well suited to CAT. For example, blood pressure (BP) readings are continuous but can be binned into ranges indicating (1) ineligible for treatment, (2) eligible for treatment, or (3) refer for hypertensive emergency. These categories then can become multinomial or binary steps and remain appropriate for CAT. However, a patient encounter for common mental illnesses might reveal mild, moderate, or severe depression, potentially comorbid with anxiety and/or post-traumatic stress disorder. The absence of mutually exclusive categories makes this step poorly suited to CAT. In addition, steps that assess quality or fidelity of delivery might be poorly suited to CAT as currently conceptualized.
CAT Data Sources
Ideal routine health information attributes include validity, timeliness, completeness, representativeness, and reliability.3 Data sources for populating CAT should additionally be aggregated correctly, physically accessible, represent denominators of interest, and not add additional burden to HCWs. Unsurprisingly, it is rare to find a singular data source that meets all of these qualities. Adapting CAT to a new cascade requires determining which data sources are “good enough” and which services they represent. This process requires a methodical approach; we have used data mapping procedures to identify possible data sources and note their strengths and limitations (Table 1). Data mapping procedures emerged from the related process mapping; instead of patient flow, they illustrate where data are collected, transported, recorded, and stored. Predetermined rules for data quality have been useful in some CAT adaptations, often following guidance from the Global Fund that uses a 10% error rate as a threshold above which data quality improvement activities should be undertaken before introducing CAT. In some cases, adequate data systems do not exist and must be created. This may be more common in low-resource settings where health systems are moving from acute to chronic care service models.
CAT has been adapted to diverse cascades globally, to communicable and noncommunicable diseases (NCD), and to single encounters and chronic care management. Evaluations are ongoing to assess the impact of combination interventions incorporating CAT on health and implementation outcomes, using qualitative and quantitative methods. A core research team supported CAT adaptation, but scientific leadership was unique across the adaptations. This section outlines CAT adaptations (Table 2).
A mobile smart phone PCAT application was developed (mobile PCAT [“mPCAT”] [NIH R21AI124399])13 to address the computer access and computer literacy barriers of the Excel-based PCAT. Mobile phones are widely used in Mozambique and Kenya, and mPCAT's development was driven by nurses' desire to independently use data to gauge service flow and identify bottlenecks. During usability testing and feasibility pilots in Kenya and Mozambique, several adaptations were made, including simplification of data entry, harmonization of labels with national registry norms, and ensuring easy sharing of results through WhatsApp or email.13 Currently, the mPCAT is being used in SAIA-SCALE (R01MH113435),14 which is scaling-up SAIA across one Mozambican province, and is available on Google Playstore as PCAT Mozambique and PCAT Kenya.
Pediatric and Adolescent HIV
The second CAT adaptation was to the pediatric and adolescent HIV cascade in Kenya (F32HD088204; R34AI129900), the “PedCAT.” The PedCAT includes 5 steps: (1) HIV testing uptake, (2) linkage to care, (3) ART initiation, (4) viral load (VL) monitoring, and (5) VL suppression. Data sources aggregated to reflect children (0–9 years), adolescents (10–19 years), and young adults (20–24 years) distinctly were lacking. Three data sources were considered: (1) monthly aggregated, clinic-level count data from the District Health Information Service; (2) abstracted individual-level, cross-sectional facility paper registers, and (3) individual-level, longitudinal or cross-sectional electronic medical records. No single data source was sufficient to cover the 5 cascade steps. Abstraction of paper registers was selected as the only feasible approach, and a minimum of 8 registers are required to fill the PedCAT appropriately.
Family Planning and HIV Testing Integration
The third CAT adaptation modeled the integration of HIV testing into family planning (FP) services in Kenya (K24HD088229), the “FPCAT.” National guidelines recommend HIV testing in FP; in 2008, HIV testing questions were added to the FP clinic register, which contains the required inputs for the FPCAT: (1) HIV counseling and (2) HIV testing.
Hypertension and HIV Care Integration
The fourth CAT adaptation focused on integrating hypertension services into HIV care in Mozambique (R01HL142412), the “HCAT.” In many resource-limited settings, chronic care models for NCD do not exist at scale, while the HIV chronic care platform is broadly implemented. Integrating hypertension services into the HIV platform represented an opportunity to standardize and scale NCD services. The HCAT took a cross-sectional approach and included the following steps: (1) BP screening, (2) diagnosis, (3) linkage to care, (4) treatment initiation, (5) ongoing BP monitoring, and (6) controlled hypertension.
The fifth CAT adaptation focused on optimizing task-shared outpatient mental health care for epilepsy and schizophrenia in Mozambique (R21MH113691), the “MHCAT.”15,16 In contrast to previous sequential cascades, chronic outpatient mental health care involves a complex series of repeated, cyclical treatment and intervention protocols, and steps may differ by patient symptoms, diagnoses, and within patients over time. A challenge was determining which cascade steps were most important for monitoring outpatient cascade performance, while being identical for all patients within each diagnosis. The resultant MHCAT cascade steps are high-level and lose much of the granularity of detailed outpatient mental health treatment protocols. This has the benefit of identifying improvement opportunities that may be common to multiple systems, but the disadvantage of losing the opportunity to identify protocol-specific improvement opportunities.
Opioid Overdose Reversal
The sixth CAT adaptation focused on optimizing integration of naloxone—an opioid antagonist that reverses opioid overdose—distribution within syringe access programs for people who inject drugs in California, USA (R21DA046703), the “NCAT.” Like the PCAT, the NCAT has 2 cascades: naloxone distribution and community-based overdose reversal. Cascade 1 consists of 4 steps: (1) present for syringe access services, (2) trained to administer naloxone, (3) possess naloxone, and (4) immediate access to naloxone. Cascade 2 contains 3 steps: (1) overdoses observed, (2) community-based reversal attempted, and (3) successful reversals. This adaptation is the only application of CAT in a resource-rich country and in a community-based, as opposed to health facility–based, setting. The absence of robust data systems, a stigmatized health service, challenges with loss to follow-up, and limited resources available for service delivery are similarities between this adaptation and others in resource-limited settings outside of the United States. However, unique opportunities of this adaptation include having flexibility to modify data sources and markedly change service delivery organization without liaising with centralized bodies, such as ministries of health.
Cervical Cancer Screening
The seventh adaptation of CAT considers integration of cervical cancer screening into FP clinics in Kenya, the “CCS-CAT,” and is under development. Key challenges in this adaptation include properly identifying FP clients who have been screened within the recommended timeframe by age and HIV status and therefore do not require screening17 and properly categorizing women who receive screening at subsequent, rather than first, visits.
Challenges Encountered During CAT Adaptation
Challenges and design choices broadly fell into 8 categories (Table 3): (1) periods for data aggregation and analysis, (2) handling low patient volumes, (3) use of cross-sectional versus cohort-based populations, (4) integrating with electronic health records or mobile platforms, (5) analyzing a full catchment area versus patients presenting for care, (6) estimating denominators, (7) managing appropriate exit and re-entry of groups, and (8) expanding CAT operators. Examples of each are described below.
The NCAT required modification of periods (1) as, unlike in facility-based chronic care where visits typically occur at fixed intervals, syringe access visits may occur multiple times per month depending on client needs. To address this, NCAT cascade 1 was aggregated weekly. By contrast, in NCAT cascade 2 and the MHCAT, the period was extended to 3 months. Using monthly data for individual MHCAT diagnoses and for overdose reversals caused “zero” cells due to small numbers; (2) 3-month data overcame this challenge, but are less sensitive to monthly CQI cycles.
The MHCAT was the only adaptation to use a cohort-based approach (3). This adaptation required creating a unique form to track patient visits over time. An open-source electronic medical record software (OpenMRS) was used to automatically populate the MHCAT (4). Notably, the model of using individual-based cohort data creates a large data collection, review, and entry burden.
The PCAT is able to estimate the number of expected pregnancies in each clinic's catchment area and the estimated number not presenting for ANC, making it well suited to evaluate both demand generation and supply side interventions (5). However, several others lack this step. For example, for the NCAT and MHCAT, epidemiologic data on population prevalence of targeted conditions are lacking; for the PedCAT, estimates of pediatric and adolescent HIV prevalence are not available below the county level. These CAT were limited to those presenting for care, making them unsuitable to evaluate demand generation interventions.
Sources of denominator data, even within the population presenting for care, can be challenging (6). For the HCAT and MHCAT, no patient tracking forms were in use and new registers had to be developed and piloted. For the PedCAT, estimating the number of children and adolescents requiring a VL sample was complex. Abstraction from patient files to determine when individual patients' routine VLs were expected according to Kenyan guidelines was unfeasible; a mathematically calculated pseudodenominator was chosen to overcome this challenge and extend the PedCAT through to a clinically relevant health outcome.
In the FPCAT and PedCAT, testing schedules for different Kenyan populations required determining appropriate exits from the cascade for individuals not requiring testing (7). In the FPCAT, exit was required for known HIV-positive clients or those with a recent HIV test (3-monthly for key populations and annually for general population). However, this population characterization was not routinely documented in the FP register, requiring modification of data systems. By contrast, the NCAT cascade 1 has 2 different streams of services, depending on whether syringe access clients received an initial training; clients who have are not exited but have separate cascade steps. This creates 2 streams that run in parallel and intersect for the final cascade step.
The NCAT was the first to reflect community-based delivery and data collection (8). Although data sources existed to partially populate the NCAT, data collection systems were lengthy and inhibited service provision. Shortened tools are under development.
Through the first HIV care CAT, the original SAIA PCAT, and 7 subsequent adaptations, we have learned lessons on CAT structure, mathematics, acceptability, feasibility, and usability. We have encountered technical challenges (e.g., absence of appropriate data sources or complex care patterns not easily dichotomized) and created technical innovations (e.g., mobile-based collection or EMR integration).
Additional opportunities remain for CAT adaptation, such as incorporation of nonbinary outcomes, integration into e-platforms, and expansion of CAT operators. Quality of health services is a new frontier,18–20 but is often multidimensional and on a continuous scale, and not easily summarized by the binary CAT steps. Additional work is needed to incorporate quality and other nonbinary steps into CAT, while maintaining the tool's attractive simplicity. As CAT is applied to chronic care models, repeat visits within individuals need to be modeled, as well as complex case management patterns to address comorbidities, side effects, treatment switches, mortality, and migration. As electronic health information systems become more common, innovation is needed to integrate CAT into routine eHealth platforms (eg, DHIS II or OpenMRS). As CAT is scaled up, as in SAIA-SCALE, or decentralized to community-based organizations, as in SAIA-Naloxone, questions remain about who should populate the tool—HCWs, community workers, or research staff. In addition, further research is needed to identify settings in which CAT is most and least useful. For example, in the original SAIA trial, HCWs in Cote d'Ivoire (where HIV prevalence is lower and zero counts were common) found the tool to be less useful than Mozambican and Kenyan HCWs. Contextual factors beyond facility volume may impact acceptability and utility of CAT. Finally, data collection is underway to determine whether CAT is time saving for HCWs and whether stakeholders have interest in expanding and sustaining CAT use beyond research settings.
There are important epidemiologic questions about using CAT to evaluate impact in a research context, balancing specificity and usability. Although CAT is intended as an implementation tool, it could be used to populate charts for CQI evaluation, or CAT step performance could be tested using classical statistical hypothesis testing. However, there may be a tension between acceptability and specificity if CAT is to be used for both priority setting and impact evaluation. The target population to assess impact might be more homogeneous, exclude patients known to have been exposed to the intervention for longer periods, or otherwise restrict the patient population. Using CAT to evaluate impact is further complicated by the multiple steps and therefore multiple potential outcomes; initial work has investigated the discriminating power of comparing high and low performance of individual versus composite indicators in the PMTCT cascade,21 an analysis that should be replicated for other cascades. In addition, CAT includes both absolute numbers and proportions; improvements in upstream steps of CAT could result in a false sense of worsening downstream effects if only considered on the relative proportion scale rather than the absolute scale.
The use of CAT in the SAIA trial and subsequent adaptations suggest that CAT is an effective implementation strategy, although evaluations of adaptations to determine impact on health and implementation outcomes are ongoing. Although CAT was developed within the SAIA package, and the examples demonstrate its role within SAIA, CAT may be used independently or in conjunction with other implementation strategies. CAT has high acceptability and feasibility across resource-limited and high-income settings and with various HCW cadres. CAT integrates well with CQI and process mapping, two evidence-based implementation strategies,22 and may be a low cost addition to enhance their effectiveness. Further research is ongoing to examine other implementation outcomes associated with use of CAT, such as penetration and sustainability.
CAT is intended to overcome a well-described barrier between routine data availability and its use for data-informed systems optimization.23–25 It is a powerful implementation strategy that synergizes to enhance the effectiveness of well-established implementation strategies, such as CQI and process mapping, by allowing frontline HCWs to view their systems holistically and support prioritization. CAT is adaptable to a wide range of health services, populations, and settings; innovations in technology have made CAT more acceptable and usable to widespread audiences. Future methodologic challenges include simplifying data collection, reflecting more complex care systems, and scaling this approach to routine program implementation.
1. Gimbel S, Chilundo B, Kenworthy N, et al. Donor data vacuuming in global health partnerships. Med Anthropol Theor. 2018;5:79–99.
2. Gloyd S, Wagenaar BH, Woelk GB, et al. Opportunities and challenges in conducting secondary analysis of HIV programmes using data from routine health information systems and personal health information. J Int AIDS Soc. 2016;19:20847.
3. Wagenaar BH, Sherr K, Fernandes Q, et al. Using routine health information systems for well-designed health evaluations in low- and middle-income countries. Health Policy Plan. 2016;31:129–135.
4. UNAIDS. Fast-track: Ending the AIDS Epidemic by 2030. 2014. Available at: http://www.unaids.org/sites/default/files/media_asset/JC2686_WAD2014report_en.pdf
. Accessed April 29, 2019.
5. Hamilton E, Bossiky B, Ditekemena J, et al. Using the PMTCT cascade to accelerate achievement of the global plan goals. JAIDS J Acquir Immune Defic Syndr. 2017;75:S27–S35.
6. Chi BH, Tih PM, Zanolini A, et al. Implementation and operational research. JAIDS J Acquir Immune Defic Syndr. 2015;70:e5–e9.
7. Hargreaves JR, Delany-Moretlwe S, Hallett TB, et al. The HIV prevention cascade: integrating theories of epidemiological, behavioural, and social science into programme design and monitoring. Lancet HIV. 2016;3:e318–e322.
8. Hensen B, Fearon E, Schaap A, et al. Application of an HIV prevention cascade to identify gaps in increasing coverage of voluntary medical male circumcision services in 42 rural Zambian communities. AIDS Behav. 2019;23:1095–1103.
9. Sherr K, Gimbel S, Rustagi A, et al. Systems analysis and improvement to optimize pMTCT (SAIA): a cluster randomized trial. Implement Sci. 2014;9:55.
10. Rustagi AS, Gimbel S, Nduati R, et al. Impact of a systems engineering intervention on PMTCT service delivery in Cote d'Ivoire, Kenya, Mozambique: a cluster randomized trial. J Acquir Immune Defic Syndr. 2016;72:e76–e82.
11. Micek MA, Gimbel-Sherr K, Baptista AJ, et al. Loss to follow-up of adults in public HIV care systems in central Mozambique: identifying obstacles to treatment. J Acquir Immune Defic Syndr. 2009;52:397–405.
12. Gimbel S, Voss J, Mercer MA, et al. The prevention of mother-to-child transmission of HIV cascade analysis tool: supporting health managers to improve facility-level service delivery. BMC Res Notes. 2014;7:743.
13. Kawakyu N, Nduati R, Munguambe K, et al. Development and implementation of a mobile phone–based prevention of mother-to-child transmission of HIV cascade analysis tool: usability and feasibility testing in Kenya and Mozambique. JMIR Mhealth Uhealth. 2019;7:e13963.
14. Sherr K, Ásbjörnsdóttir K, Crocker J, et al. Scaling-up the Systems Analysis and Improvement Approach for prevention of mother-to-child HIV transmission in Mozambique (SAIA-SCALE): a stepped-wedge cluster randomized trial. Implement Sci. 2019;14:41.
15. Wagenaar BH, Cumbe V, Raunig-Berhó M, et al. Health facility determinants and trends of ICD-10 outpatient psychiatric consultations across Sofala, Mozambique: time-series analyses from 2012 to 2014. BMC Psychiatry. 2015;15:227.
16. Wagenaar BH, Cumbe V, Raunig-Berho M, et al. Outpatient mental health services in Mozambique: use and treatments. Psychiatr Serv. 2016;67:588.
17. Ministry of Health. PANGA UZAZI National Family Planning Guidelines for Service Providers. 2010. Available at: https://www.k4health.org/sites/default/files/Kenya FP Guidelines 2010 final signed_full text.pdf
. Accessed May 20, 2019.
18. Kruk ME, Gage AD, Arsenault C, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Heal. 2018;6:e1196–e1252.
19. Berwick D, Snair M, Nishtar S. Crossing the global health quality chasm: a key component of universal health coverage. JAMA. 2018;320:1317–1318.
20. World Health Organization, World Bank. Delivering Quality Health Services: A Global Imperative for Universal Health Coverage. Geneva, Switzerland; 2018.
21. Gimbel S, Voss J, Rustagi A, et al. What does high and low have to do with it? Performance classification to identify health system factors associated with effective prevention of mother-to-child transmission of HIV delivery in Mozambique. J Int AIDS Soc. 2014;17:18828.
22. Sifrim ZK, Barker PM, Mate KS. What gets published: the characteristics of quality improvement research articles from low- and middle-income countries. BMJ Qual Saf. 2012;21:423–431.
23. Monks T. Operational research as implementation science: definitions, challenges and research priorities. Implement Sci. 2016;11:1–10.
24. Wagenaar BH, Gimbel S, Hoek R, et al. Effects of a health information system data quality intervention on concordance in Mozambique: time-series analyses from 2009-2012. Popul Health Metr. 2015;13:9.
25. Wagner AD, Crocker J, Liu S, et al. Making smarter decisions faster: systems engineering to improve the global public health response to HIV. Curr HIV/AIDS Rep. 2019;16:279–291.