The successful delivery of health services to improve the health of women living with HIV and prevent mother-to-child transmission of HIV (PMTCT) is pivotal for achieving an HIV/AIDS-free generation.1 In 2014, there were an estimated 210,000 pregnant women living with HIV in Nigeria and yet coverage of PMTCT services remained low at 29%.2 Nigeria began offering lifelong antiretroviral treatment (ART) to all pregnant and breastfeeding women living with HIV regardless of CD4 cell count or clinical stage in 2016 (Option B+)—though a number of demonstration sites started in 2014.3 However, many women living with HIV are commonly lost along the cascade of PMTCT services, which reduces the likelihood of achieving viral suppression in women and increases the risk of mother-to-child transmission. Although the definition of retention-in-care is not standardized,4,5 reports from Nigeria suggest that between 62% and 76% of persons living with HIV remain in HIV-related care.6,7
Although household and community factors are important, the quality of interactions between health systems and women living with HIV are opportunities to influence a woman's likelihood to remain in care and thereby protect her health and survival.8 Suboptimal patient care and management, prolonged waiting times, drug stock outs, and insufficient trained staff all undermine patient's confidence and satisfaction and hence the effectiveness of services.9,10
Continuous quality improvement (CQI) is a health systems' intervention methodology that uses serial reviews of local facility data to test solutions to barriers for consistent high-quality service delivery.11–13 The Breakthrough Series (BTS) is a specific CQI approach that, in addition to facility-level data review and action cycles, also includes collaborations whereby staff from several health facilities meet regularly to jointly review and share ideas to seek improvements.14,15 BTS-CQI has achieved sustained improvements in other areas of patient care11,14 but has not been substantially used in the context of PMTCT services or to improve retention-in-care. We implemented a cluster randomized controlled study to examine whether a BTS-CQI intervention can improve retention-in-care, defined as attendance at 6 months postpartum while not missing previous scheduled visits, among mothers living with HIV who have started lifelong ART.
Details of the study, called Lafiyan Jikin Mata or “Excellent Health for Mothers” in Hausa, have been published4 and are summarized below.
Study Sites and Participants
Two hundred forty-one health facilities in Benue and Kaduna states in northern Nigeria, and supported by the Center for Integrated Health Programs with funding from the US Center for Disease Control and Prevention, were considered for inclusion. Facilities were eligible if they had offered PMTCT services for more than 6 months, regularly identified >1 pregnant woman per month, provided onsite delivery and postpartum care, and had at least 2 trained community health extension workers. Thirty-two facilities (20 primary and 12 secondary health level), stratified by location and level of health care, were randomly selected and allocated to either intervention or control arms using computer-generated random numbers (Fig. 1).
All pregnant women attending their first antenatal booking were offered HIV testing. Women diagnosed with HIV infection or known to be HIV infected were enrolled if the gestational age was 34 weeks or less, they were ART naive, agreed to start lifelong ART, and provided written informed consent to participate in the study. CQI training and implementation started in March 2014, enrollment started in July 2014, and 6-month postpartum follow-up of all subjects was completed in October 2016.
Meetings were convened with health managers and implementers and driver diagrams developed to understand the primary and secondary drivers for low retention-in-care among mothers living with HIV (see figure, Supplemental Digital Content 1, http://links.lww.com/QAI/A992). The main drivers for low retention-in-care were identified as long waiting times, poor quality of services, and poor uptake of services. Three areas of interaction between women and facilities were thereafter identified for improvement: time spent receiving services at facilities, attitude of health workers and client satisfaction, and access to ART on same day as Antenatal care (ANC)/postnatal clinic visits. Local Government and State level CQI teams were established to oversee the initiative.
At intervention facilities, CQI teams were established to be composed of the facility head/medical officer, study nurses/community health extension workers, and laboratory technicians. Teams agreed priority areas for improvement related to the 3 main drivers and then used the Plan-Do-Study-Act model to formulate action plans, agree on indicators, collate and analyze facility data, and thereby test ideas for improving these specific areas. CQI coaches visited every 2 weeks to guide implementation of change ideas, including how to measure the process and indicators of change ideas being tested, plot run-charts and provide structured assessment and response tools16; technical assistance was also provided by phone. Facility CQI teams developed simple “run charts” (line graphs used to identify and display trends over time)17,18 to track changes in performance over time as change ideas were implemented and tested. Changes were then adopted, adapted, or abandoned by the facility staff based on the trends or shifts of the run charts. For example, one facility tested the use of a staff “clock-in” register to stimulate early attendance of staff and reduce time spent by clients in the facility. Weekly run-charts presented the times when staff were in attendance alongside average times of clients to access services (Fig. 2). The run charts showed that waiting time reduced when staff started their work earlier; as a result, “clock-in” registers were adopted.
Facility teams also met together in quarterly BTS collaborative learning sessions.14
During the BTS collaborative sessions, teams shared learning experiences of change ideas implemented, examples of run charts and progress including how they proposed to adapt and implement changes. Additionally, facility CQI teams regularly communicated with each other by telephone and cross-site visits to strengthen learning and exchange of ideas.
Facilities randomized to the control arm received the routine support as specified in the Nigeria Ministry of Health guidelines and supervision.19
General PMTCT Program Support
All sites received technical assistance every 3–6 months from Center for Integrated Health Programs team including the following: Data Quality Assurance assessments that reviewed the accuracy, completeness, and reliability of facility data20; Standard of Care reviews that evaluated service quality using predefined indicators; and Model of Care reviews that assessed the system, structure, and procedures at site level.20
Routine facility data were abstracted from clinic registers to determine adherence to appointments and calculate rates of retention-in-care. In addition, descriptive data related to health facilities and study participants were documented. At all facilities, client satisfaction and waiting times were recorded at the beginning and end of the study. Ten women (not necessarily study participants) attending antenatal clinics at each site were systematically included; if less than 10 women attended on the day of assessment, then all women were included. Pregnant women were asked to complete a 12-point client satisfaction questionnaire with Likert rating scales.21 Waiting time from arrival at the clinic until last service received was documented. In addition, intervention facilities collected real-time data on waiting times (weekly or monthly depending on the change idea) and used these data to develop run charts to track the effect of the idea being tested (Fig. 2). Date of visits by CQI coaches and collaborative meetings were routinely recorded and evaluated against planned schedules.
Unanticipated industrial strike actions and civil unrest occurred several times during the study period. Calendar months when facilities were unable to provide routine service delivery for at least 1 day were identified. These data were used to estimate exposure to disrupted services for (1) individual women (months with disrupted services during an individual woman's enrolment/months of individual enrolment) and (2) facilities (months with disrupted services/months of study implementation).
Primary and Secondary Outcomes
There primary outcome was retention-in-care at 6 months postpartum. A study participant was regarded as:
- fully retained-in-care at 6 months postpartum if the woman attended the 6-month postpartum visit (±30 days) and did not miss any previous scheduled visit by more than 30 days (starting from ANC booking).
- partially retained-in-care at 6 months postpartum if the woman attended the 6-month postpartum visit (±30 days) but missed 1 or more earlier scheduled visits by more than 30 days.
- not retained in care at 6 months postpartum if the woman did not attend the 6-month postpartum visit (±30 days), regardless of whether she attended or missed any other scheduled visit.
The was no national standardized schedule of ART visits and therefore the date of next scheduled visit as given by the health worker for the individual client was used to determine adherence and therefore the primary outcome measure. This was defined as the documented date given by the health worker for the woman to come for her next appointment, including ARV drug pick-up.
Secondary outcomes were the proportion of HIV-exposed infants born to enrolled mothers who: (1) started antiretroviral (ARV) prophylaxis within 72 hours of birth; (2) received co-trimoxazole prophylaxis at 6–10 weeks of age; (3) had a dried blood spot (DBS) collected for early infant diagnosis (EID) of HIV between 6 and 10 weeks of age; and (4) proportion of pregnant women living with HIV initiating ART within 2 weeks of enrolment.
Women were considered to have “transferred out” if there was documentation they had explicitly relocated.
Sample Size Estimation
No data on rates of retention-in-care based on a standardized definition to inform estimates were available from Nigeria. We therefore used program data on the percentage of pregnant women who missed their last 3 months' appointment to inform assumptions. Based on these data, we assumed baseline retention-in-care to be 40% at 6 months postpartum.
The sample size was calculated, using the Windows Program for Epidemiologists,22 to be able to detect a 20% absolute difference (ie, 60% vs. 40%) with a 95% significance level (2-sided) at 80% power. The minimum sample size needed under individual randomization to detect this absolute difference was 94 participants for each group (intervention and control). This sample size was inflated to account for intrasite variability. An inflation factor (IF) was calculated as follows: IF = 1 + (n − 1) ro, where n is the number of participants recruited at each site and ro is the internal variability within each site. Ro was assumed to be equal to 0.10; assuming a total number of 16 clusters, the sample size had to be inflated by a multiple of 2.9. A minimum sample of 272 participants was therefore estimated; however, allowing for 20% incomplete data or attrition, an estimated sample size of 327 pregnant women living with HIV being enrolled per study arm was agreed.
Descriptive statistics were used to summarize facility and individual characteristics and cluster-adjusted χ2 test and t test analyses were used to determine associations. The effect of the intervention on primary and secondary outcomes was examined through an intention-to-treat analysis that included all enrolled women except those who withdrew their consent. Effects were modeled using log binomial regression with generalized estimating equation with an independent correlation structure and robust standard errors. Crude and adjusted odds ratios were obtained from the generalized estimating equation as a measure of association, adjusting for explanatory variables. Statistical significance was set at 0.05. All statistical analyses were performed using STATA 14.23
A per-protocol analysis was also conducted in which women who died; whose pregnancy resulted in stillbirth, miscarriage, neonatal, or infant death; or who transferred out to another facility were excluded.
The study protocol and amendments were approved by the Nigerian National Health Research Ethics Committee and the World Health Organization Ethics Review Committee, Geneva.
Twenty-six facilities—14 intervention and 12 control—contributed data to the analyses (Fig. 1). Six facilities were excluded because less than 2 pregnant women living with HIV were recruited during the enrolment period. Twenty-one women withdrew from the study, leaving a total of 511 women eligible for analyses.
Baseline characteristics of health facilities and study participants are detailed in Table 1. The median age of women at enrollment was 27 years (interquartile range, 23–50 years). Nearly all were married and more than half had at least a secondary education. All government-owned health facilities experienced service disruption for about 20% (interquartile range, 15%–40%) of the study period (see table, Supplemental Digital Content 2, http://links.lww.com/QAI/A992).
Overall, 441 of 511 participants (86%) were in care at the time of delivery and 311 of 511 participants (61%) attended their 6-month postpartum clinic appointment; 155 of 511 participants (30%) missed at least 1 scheduled visit before 6 months postpartum by more than 30 days (Table 2). About 71% (364/511) of women attended the clinic for a scheduled visit within the 90 days before the 6-month postpartum visit. There were no differences in these patterns by study arm based on national definition.
According to study definitions, 43% (219/511) of participants were fully retained at 6 months across both arms (ie, attended 6-month visit and did not miss any previous scheduled visit); there was no significant difference between intervention (44%) and control facilities (41%) [adjusted relative risk (ARR) = 1.08; 95% confidence interval (CI): 0.78 to 1.49] (Table 3), and there was similarly no effect on the distribution of participants between full, partial, and not retained. This remained true even after adjusting for women's exposure to service disruptions and facility strike score (Table 3). There were no differences in any measure of retention in the per-protocol analysis (see tables, Supplemental Digital Content 3–5, http://links.lww.com/QAI/A992).
Secondary Outcome Analysis
Nearly all women were started on ART within 2 weeks of enrolment in both intervention (98.9%) and control (94.3%) arms (Table 2). Infant ARV prophylaxis initiation within 72 hours was not different by study arm (66% intervention arm vs. 74.7% control, ARR = 0.95; 95% CI: 0.84 to 1.07) (Table 3 and Supplemental Digital Content 6, http://links.lww.com/QAI/A992) although it was significantly associated with place of delivery, that is, 51% more likely among infants born in health facilities compared to those that were born elsewhere (78% vs. 22%, ARR = 1.51; 95% CI: 1.20 to 1.91) (Table 3). DBS collection within 6–10 weeks of age was only 37.5% overall but was more likely among infants born to women attending intervention health facilities compared to those in control health facilities (48.8% vs. 25.3%, ARR = 1.76; 95% CI: 1.27 to 2.42) (Table 3). DBS collection at 6–10 weeks of age was also positively associated with women who had secondary or higher education (relative risk = 1.96; 95% CI: 1.50 to 2.57) (see table, Supplemental Digital Content 7, http://links.lww.com/QAI/A992). The overall proportion of infants started on CTX between 6 and 10 weeks of age was 56.6% (228/403) but the likelihood of CTX initiation was no different by study arm though it was slightly greater for infants born in facilities compared to those delivered outside the facilities (ARR = 1.30; 95% CI: 1.03 to 1.64) (Table 3 and Supplemental Digital Content 8, http://links.lww.com/QAI/A992).
Intermediate CQI Indicators
A full description of CQI process indicators are provided in a linked publication.24 Four indicators of intermediate CQI indicators and process indicators are presented to illustrate adherence to the CQI protocol and responses at health facilities. Before CQI-BTS implementation, the time spent by pregnant women at ANC services (all women attending ANC, not just those living with HIV) was longer at intervention sites compared to control sites [median 182 (range: 22–389) vs. 139 (range: 52–229), respectively]. At the end of the study, time spent by pregnant women for receiving services was less at intervention facilities than at control sites [median 125 (range: 40–270) vs. 201 (range: 43–406), respectively]. Facility-specific run charts of weekly assessments illustrate improvements at individual sites (Fig. 2).
The mean difference in time spent by patients receiving ANC services between baseline and end of study reduced in the intervention arm by 54 minutes (95% CI: −67 to −40), whereas in the control arm it increased by 58 minutes (95% CI: 44 to 73). After adjusting for facility characteristics, there was a significant difference in the change in time spent by pregnant women receiving ANC services between baseline and end of study in the intervention compared to control health facilities (adjusted mean difference: −110 minutes; 95% CI: −128 to −92).
However, average client satisfaction assessment score for ANC attendees was no different between intervention and control sites before, and at the end of the study period (baseline: 47.2 vs. 46.7; end evaluation: 48.4 vs. 47.9; adjusted mean difference, −0.33; 95% CI: −2.27 to +1.62).
The majority of facility-based CQI activities and collaboratives took place as scheduled: 57 of 65 (88%) and 38 of 53 (72%), respectively.
The CQI intervention implemented as part of the study did not result in any significant difference in rates of retention at 6 months postpartum among mothers living with HIV. However, the proportion of HIV-exposed infants from whom a DBS was collected between 6 and 10 weeks of age for infant diagnosis was 75% greater in the intervention arm, and waiting times among pregnant women attending ANC were also significantly less at intervention sites. These suggest that staff at intervention facilities did engage with the CQI process and were willing to implement at least some changes in daily routines that led to measurable improvements in services.
There are several possible explanations for the lack of observed effect on our primary outcome of retention-in-care: the underlying theory of change may have been incorrect; for example, the areas of service delivery targeted for change were too indirectly related to the outcome of interest and not likely to improve retention-in-care among women; the duration, dose, or intensity of the intervention was insufficient or the CQI intervention was not satisfactorily executed25; the health system was resistant to the intervention; factors extraneous to the health system overwhelmed any effects of improved service delivery on the decision-making processes and practices of mothers living with HIV.26 The rates of retention-in-care in the study population were also lower than what had been reported previously in Nigeria.6,7 This probably reflected our more stringent study definition of “full retention” (seen at 6-month postpartum visit and at ALL intermediate follow up visits) than the general programmatic definition (“had a visit within 90 days before 6-month postpartum visit”), differences in measurement approach, and also possible selection bias as both study arms received PEPFAR support.
On reflection, our field teams considered that the overall intervention time for the study was not sufficient to produce significant changes in practice among health workers and thereby impact on the attendance practices of mothers and their infants. We were unable to evaluate if there was a difference in effect of the intervention among women recruited early in the study compared to those who were recruited later. Health workers required time to assimilate the concept and working of a CQI approach and to become familiar and confident with using data for improvement. There had not been sufficient time to implement and institutionalize the CQI processes before enrolment to the study began. Although most of the planned CQI activities took place as scheduled, it was challenging to implement the CQI data review and action cycles in a matter of weeks or months. The process required high-quality reliable data to be generated to serve as a basis for action and planning by staff at individual clinics. Again, CQI coaches reported at research team meetings that this proved challenging to achieve in clinics, even if individual health workers were committed in principle and some feared that low performance data might be seen as a negative reflection on their commitment to patients.
Option B+ was introduced in the states where the study was implemented before adoption nationally around the same time as the study was launched and this required time for facilities to adapt to new protocols. Clinics were also subjected to a series of service disruptions, including strikes, civil unrest, and national elections. Given that CQI depends on regular cycles of review and action, it is possible that continuity and momentum around the changes introduced through CQI were lost with each recurring disruption. It is also possible that the technical support through PEPFAR-funded comprehensive care and treatment programs to all sites may have improved the quality of services everywhere and blunted the effects of the CQI intervention. Our CQI coaches found that while health workers could improve some local factors impeding service delivery, other systemic challenges such as insufficient staff and weak infrastructure27,28 could not be remedied by local staff but would have required the intervention of local government health managers.
The CQI intervention was, however, associated with improved infant DBS collection at 6–10 weeks of age. Historically, EID service provision in Nigeria has been very challenging because of weak health systems and shortages of skilled personnel.29 Although EID was not a specific driver for retention-in-care identified by the facility-based teams, it was considered an indicator of quality of services and utilization and the same facility quality improvement teams did review these data on a regular basis, which may have resulted in the change. It may be that a specific, relatively short-term target such as increasing DBS collection for EID may be more amenable to rapid interventions than a more distant target of retention-in-care at 6 months or longer that is also subject to many other community and personal influences.
Even though the intervention did not significantly improve rates of maternal retention-in-care, the study provides data to inform other efforts to improve coverage and uptake of HIV interventions. Almost all participants (>94%) across study arms initiated ART within 2 weeks of diagnosis as a result of the “test and start” protocol that was beginning to be piloted at our sites. This shows a high level of acceptance and readiness by pregnant women with HIV to commence ART upon diagnosis.30 In addition, HIV-exposed infants were more likely to receive co-trimoxazole prophylaxis at 6–10 weeks and initiate ARV prophylaxis within 72 hours of birth if their mothers delivered in the hospital. These findings are not surprising and provide continued impetus to promote facility deliveries. The uptake of CTX was low across all sites and did not differ by study arm.
To enable Nigeria to meet the 90% target of reductions in new infant HIV infections and successfully roll out option B+, the full PMTCT cascade needs to be achieved for mother–infant pairs and sustained across the public health sector. The health benefits of option B+ for women living with HIV are dependent on high levels of retention-in-care and adherence to ART. Both require health systems that are reliable and able to respond to the needs of the population they serve. CQI has been shown to improve the quality of service delivery elsewhere. Although there were “signals” that staff were receptive to the CQI process and significant improvements in EID testing occurred, the CQI intervention implemented as part of the study did not result in major changes in rates of retention-in-care among mothers living with HIV. Instead, the study highlights that there are no easy, quick fixes to large systems that have been fragmented over many years and still exposed to many disruptions. Health systems are the platform for quality of care services including all that is needed for follow-up of women living with HIV. Significant long-term investment, leadership, and commitment are needed to bring about improvements, and CQI, as a feasible and locally driven innovation and response, merits further study to support these transformations.
The Center for Integrated Health Programs expresses profound gratitude to Global Affairs Canada; World Health Organization Geneva and AFRO (especially Nigel Rollins, Nita Bellare, Shaffiq Essajee, Taiwo Oyelade, Francoise Bigirimana, and April Baller); Meade Morgan; The Institute for Healthcare Improvement (IHI), USA and Ubora Institute, Ghana; Federal Ministry of Health Nigeria; and Kaduna and Benue State Ministries of Health for supporting the work on strengthening health systems and improving health outcomes for pregnant women living with HIV and their infants in Nigeria.
1. World Health Organization. PMTCT Strategic Vision 2010-2015: Preventing Mother-to-child Transmission of HIV
to Reach the UNGASS and Millennium Development Goals: Moving towards the Elimination of Paediatric HIV
, December 2009. 2010. Available at: http://apps.who.int/iris/bitstream/10665/44268/1/9789241599030_eng.pdf. Accessed January 30, 2017.
2. UNAIDS. Progress Report on the Global Plan. 2015. Available at: http://http://www.unaids.org
/sites/default/files/media_asset/JC2774_2015ProgressReport_GlobalPlan_en.pdf. Accessed January 30, 2017.
3. UNAIDS. Global Plan Country Factsheet_Nigeria. 2016. Available at: http://http://www.unaids.org
/sites/default/files/media/documents/UNAIDS_GlobalplanCountryfactsheet_nigeria_en.pdf. Accessed February 28, 2017.
4. Oyeledun B, Oronsaye F, Oyelade T, et al. Increasing retention in care of HIV
-positive women in PMTCT services through-breakthrough (CQI-BTS) series in primary and secondary health care facilities in Nigeria
: a cluster randomized controlled trial. The Lafiyan Jikin Mata Study. J Acquir Immune Defic Syndr. 2014;67(suppl 2):S125–S131.
5. Rollins NC, Becquet R, Orne-Gliemann J, et al. Defining and analyzing retention-in-care
among pregnant and breastfeeding HIV
-infected women: unpacking the data to interpret and improve PMTCT outcomes. J Acquir Immune Defic Syndr 2014;67(suppl 2):S150–S156.
Country Operational Plan (COP) Strategic Direction Summary. 2016. Available at: https://http://www.pepfar.gov
/documents/organization/257635.pdf. Accessed January 30, 2017.
7. Babatunde O, Ojo OJ, Atoyebi OA, et al. Seven year review of retention in HIV
care and treatment in federal medical centre Ido-Ekiti. Pan Afr Med J. 2015;22:139.
8. Myer L, Phillips TK, Zerbe A, et al. Optimizing antiretroviral therapy (ART) for maternal and child health (MCH): rationale and design of the MCH-ART study. J Acquir Immune Defic Syndr. 2016;72:S189–S196.
9. Okoli JC, Lansdown GE. Barriers to successful implementation of prevention-of-mother-to-child-transmission (PMTCT) of HIV
programmes in Malawi and Nigeria
: a critical literature review study. Pan Afr Med J. 2014;19:154.
10. Taylor NK, Buttenheim AM. Improving utilization of and retention in PMTCT services: can behavioral economics help? BMC Health Serv Res. 2013;13:406.
11. Singh K, Speizer I, Handa S, et al. Impact evaluation of a quality improvement
intervention on maternal and child health outcomes in Northern Ghana: early assessment of a national scale-up project. Int J Qual Health Care. 2013;25:477–487.
12. Youngleson MS, Nkurunziza P, Jennings K, et al. Improving a mother to child HIV
transmission programme through health system redesign: quality improvement
, protocol adjustment and resource addition. PLoS One. 2010;5:e13891.
13. Langley G, Moen R, Nolan K, et al. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. San Francisco, CA: John Wiley & Sons; 2009: 512.
14. The Breakthrough Series—IHI's Collaborative Model for Achieving Breakthrough Improvement. Available at: http://http://www.ncpc.org.uk
/sites/default/files/Anita_IHIBreakthroughSerieswhitepaper2003.pdf. Accessed 28 February 28, 2017.
15. Øvretveit J, Bate P, Cleary P, et al. Quality collaboratives: lessons from research. Qual Saf Health Care. 2002;11:345–351.
16. Centre for Integrated Health Programs CIHP. LJM Assessment and Response Tools. 2015. Available at: http://http://www.cihpng.org
/wp-content/uploads/2017/02/LAFIYAN-JIKIN-MATA-DATA-COLLECTION-TOOLS.pdf. Accessed February 28, 2017.
17. Daudelin DH, Selker HP, Leslie LK. Applying process improvement methods to clinical and translational research: conceptual framework and case examples. Clin Transl Sci. 2015;8:779–786.
18. Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011;20:46–51.
19. Federal Ministry of Health NASCP. Integrated-National-Guildlines-For-HIV
. 2014. Available at: http://emtct-iatt.org/wp-content/uploads/2016/09/Integrated-National-Guildlines-For-HIV
-2014.pdf. Accessed January 30, 2017.
20. USAID. Data Quality Standards. 2009. Available at: http://usaidprojectstarter.org/sites/default/files/resources/pdfs/TIPS-DataQualityStandards.pdf. Accessed February 2, 2017.
21. Likert R. A Technique for the Measurement of Attitudes 1932. Available at: http://http://www.socialresearchmethods.net
/kb/scallik.php. Accessed November 25, 2016.
22. Abramson JH. WINPEPI (PEPI-for-Windows): computer programs for epidemiologists. Epidemiol Perspect Innov. 2004;1:6.
23. StataCorp. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP; 2015.
24. Osibo B, Oronsaye F, Alo OD, et al. Using small tests of change to improve PMTCT services in northern Nigeria
: experiences from implementation of a Continuous Quality Improvement
and BreakThrough Series Program. J Acquir Immune Defic Syndr. 2017;75(suppl 2):S165–S172.
25. Bailie RS, Si D, O'Donoghue L, et al. Indigenous health: effective and sustainable health services through continuous quality improvement
. Med J Aust. 2007;186:525.
26. Hodgson I, Plummer ML, Konopka SN, et al. A systematic review of individual and contextual factors affecting ART initiation, adherence, and retention for HIV
-infected pregnant and postpartum women. PLoS One 2014;9:e111421.
27. Ugo O, Ezinne E-A, Modupe O, et al. Improving quality of care in primary health-care facilities in rural Nigeria
: successes and challenges. Health Serv Res Manag Epidemiol. 2016;3:1–6.
28. Welcome MO. The Nigerian health care system: need for integrating adequate medical intelligence and surveillance systems. J Pharm Bioallied Sci. 2011;3:470–478.
29. Agboghoroma C, Sagay S, Ikechebelu J. Nigerian prevention of mother to child transmission of human immunodeficiency virus program: the journey so far. J HIV
Hum Reprod. 2013;1:1–7.
30. Katirayi L, Namadingo H, Phiri M, et al. HIV
-positive pregnant and postpartum women's perspectives about Option B+ in Malawi: a qualitative study. J Int AIDS Soc. 2016;19:20919.