The Public Health Accreditation Board (PHAB) was formed in 2007 for the purpose of improving the quality and performance of state, county, tribal, and territorial health departments in the United States, with a goal to be transformative.1–3 The first health departments were accredited in 2013, and now more than 200 have achieved accreditation, with even more in the pipeline.4 Approximately 70% of the US population is currently served by a local or state health department that has received accreditation.5 Since PHAB is now in its 10th year as an organization, it is an auspicious time to examine perceptions and activities regarding the impact of PHAB accreditation on local health departments' (LHD) quality improvement (QI) and performance management (PM) activities.
There is a growing body of literature suggesting that PHAB accreditation is beneficial for health departments and potentially their communities. Davis and colleagues6 report that accreditation is a driver for adoption of a quality culture. Others have recognized it as a major motivation to conduct community health assessments and improvement plans.7 Accreditation is also associated with greater likelihood of pursuing joint community health assessments with tax exempt hospitals8,9 and for QI efforts more broadly.10 Health departments accredited for 1 year reported strengthened accountability and transparency, as well as improved management processes.5
The Public Health Accreditation Board's Domain 9 embodies specific standards and measures related to QI and PM. Standard 9.1 calls for the “Use of a performance management system to monitor achievement of organizational objectives,” seeking evidence that the following dimensions of a PM system are in place: (1) organizational objectives; (2) indicators to measure progress toward meeting the organizational objectives; (3) monitoring systems for assessing and reporting progress are in place; (4) mechanisms for identifying areas requiring focused QI processes to meet objectives; and (5) visible senior leadership supporting PM.11 The expectation for developing and implementing QI processes integrated into health department organizational practices is addressed in Standard 9.2.11 Similarly, there are several components: (1) integration of QI into staff training, organizational structures, processes, services, and activities; (2) application of an improvement model for QI efforts; and (3) support and utilization by staff at all levels of the agency. Guidance is provided regarding required documentation necessary to demonstrate conformity, and much depends upon a required written QI plan.11
The National Association of County & City Health Officials (NACCHO) regularly conducts a comprehensive national survey to profile LHDs. In addition to this profile survey that all US LHDs receive, roughly a quarter of them are randomized and requested to complete a module related to QI. The most recent profiles occurred in 2013 and 2016, with baseline data from 2010 before accreditation was launched.12–14 PHAB also collects LHD data throughout the accreditation process. Considered together, the data enable an examination of LHD quality efforts, with the potential to compare differences over time among LHDs based upon their accreditation status. Specifically, we posited 2 research questions: (1) Does applying for or achieving PHAB accreditation impact how LHDs perceive their capacity and performance of QI and PM? (2) Does applying for or achieving PHAB accreditation impact LHDs' reported activities of QI and PM?
Data from NACCHO's 2010, 2013, and 2016 National Profile of Local Health Departments (Profile) studies were used to assess trends in LHD participation in QI activities over time.12–14 The Profile questionnaire is administered via a Web-based survey sent to every LHD in the United States, but some questions (including those on QI) are included in a module that is administered to a statistical sample of LHDs. For each Profile survey, LHDs serving large populations were oversampled to ensure that sufficient numbers of them were included in the module samples. The module containing QI questions was sent to 624 (85% response rate) in 2010, 624 (79% response rate) in 2013, and 625 (80% response rate) in 2016.12–14
The 2010, 2013, and 2016 Profile questionnaires included the same question on overall level of LHD engagement in QI: “Which of the following statements best characterizes your LHD's current quality improvement activities?” Response choices were as follows: (1) LHD has implemented a formal QI program agency-wide. (2) Formal QI activities are being implemented in specific programmatic or functional areas of the LHD but not on an agency-wide basis. (3) Local health department's QI activities are informal or ad hoc in nature. (4) Local health department is not currently involved in QI activities. All three Profile questionnaires also included the question: “In the past 12 months, how many formal projects has your LHD implemented to improve the quality of a service, process or outcome?” Response options were: none; 1-3; 4-6; 7-10; 11-20; and more than 20. Responses were combined to create 2 summary variables—4 or more projects and 7 or more projects. The 2013 and 2016 Profile questionnaires included a series of 8 dichotomous questions regarding the presence of specific elements of a formal QI program (shown in Table 1). The questions about the number of projects and the elements of a formal QI program were not asked of health departments that indicated that the LHD is not currently involved in QI activities. For those respondents, the questions about QI projects and elements of a QI program were recoded to “none.”
Data from PHAB's online information system (e-PHAB) were used to classify LHDs responding to the NACCHO QI module into 3 groups based on their accreditation status on June 26, 2017: (1) LHDs that had been accredited individually or are part of an accredited centralized state integrated local public health department system (accredited); (2) LHDs that had submitted their registration in e-PHAB—either as individual health departments or as part of multijurisdictional applications—but are not accredited (in process); and (3) all other LHDs (non-e-PHAB).15
Cross-sectional analyses were conducted using SAS for 3 time points: late 2010 (prior to launch of PHAB's accreditation program), early 2013 (around the time that the first LHDs achieved accreditation), and early 2016. These analyses compared the level of QI implementation, the number of QI projects, and the presence of specific elements of a formal QI program for the 3 groups of LHDs defined by accreditation status as of June 26, 2017.
Odds ratios were calculated using maximum likelihood estimation logistic regression for 4 dependent variables: (1) LHD has implemented a formal QI program agency-wide; (2) LHD had implemented 4 or more QI projects; (3) LHD has put in place 6 to 8 of the elements of the formal agency-wide QI program; and (4) agency performance data are used on an ongoing basis to drive improvement efforts. The first 2 models were run for 2010, 2013, and 2016, while the latter 2 were run only in 2013 and 2016, as the question was not asked in 2010. Each model controlled for governance (state, local, or shared) and population size (<50 000, 50 000 to <500 000, and ≥500 000). Cross-sectional analyses used estimation weights provided in the Profile data sets to account for oversampling of large LHDs.
Longitudinal analyses were conducted using Stata for a set of 116 LHDs that responded to the QI module in both 2013 and 2016. Three dichotomous variables were defined to indicate the presence or absence of certain changes in the LHD's report of the status of QI activities between 2013 and 2016: (1) any increase in reported overall level of LHD engagement in QI; (2) increase in reported overall level of engagement in QI to the highest level (formal, agency-wide QI); and (3) increase in the number of the 8 specific QI elements present at the LHD. Because of the very small number of LHDs in this longitudinal data set that were registered in e-PHAB but not yet accredited, the in process and accredited LHDs were combined into a single group (e-PHAB LHDs). Variables were individually analyzed using logistic regression to determine the odds ratios and 95% confidence intervals for the presence of these changes as a factor of an LHD's involvement in e-PHAB. Estimation weights were developed for the longitudinal analysis to account for the even greater representation of large LHDs in the matched set. The Institutional Review Board of Florida State University determined that the study did not fit the definition of human subjects and waived jurisdiction.
Unweighted statistics reflecting jurisdiction size, governance type, and PHAB status as of June 26, 2017, are presented in Table 2 for each survey year, as well as for the matching subset of LHDs from the 2013 and 2016 Profiles that are used in the longitudinal analysis. Distributions of jurisdiction size, governance, and PHAB status were similar for the 3 Profile surveys. Compared with the individual survey waves, a greater proportion of LHDs in the 2013-2016 paired subset served large jurisdictions and were accredited.
The distribution of responses to the question “Which of the following statements best characterizes your LHD's current quality improvement activities?” is presented in Table 1 by accreditation status in June 2017 for each of the 3 waves of the NACCHO Profile. In 2010, prior to the launch of PHAB accreditation, health departments that were accredited as of June 2017 reported a greater engagement in QI activities. For example, 30.1% of the accredited health departments reported that their LHD “implemented a formal quality improvement program agency-wide,” compared with 9.9% and 13.5% for health departments that are in process and not registered in e-PHAB, respectively. By the time of the 2016 Profile, the gap between accredited health departments and those that have not yet completed the registration process in e-PHAB had increased. For example, 79.1% of accredited LHDs reported formal QI agency-wide in 2016, compared with 18.4% of those not registered in e-PHAB. The gap between LHDs that are in process and those not registered in e-PHAB also widened over time, with 50.1% of in-process LHDs reporting formal QI agency-wide in 2016. This is also displayed in the Figure for visual comparison. Among the LHDs not in e-PHAB, there was also a modest decrease in LHDs reporting that they are not involved in QI activities, from 18.0% in 2010 to 13.3% in 2016, and a corresponding increase in LHDs reporting a formal QI program agency-wide, from 13.5% in 2010 to 18.4% in 2016.
When 2 response options were combined (data not shown) “formal QI program agency-wide” and “formal QI activities are being implemented in specific programmatic or functional areas of the LHD, but not on an agency-wide basis,” differences based on accreditation status become more pronounced over time. In 2010, a similar percentage of in-process LHDs (47.9%) and those not in e-PHAB (41.4%) indicated that they had some formal QI. These numbers changed to 76.5% and 49.6% by 2013, and 87.1% and 45.8% in 2016, for in-process LHDs and those not in e-PHAB, respectively. During the same interval, accredited LHDs moved steadily toward greater QI penetration within their agencies.
Examining responses to the question: “In the past 12 months, how many formal projects has your LHD implemented to improve the quality of a service, process or outcome?” there is a consistent decrease in health departments reporting no formal QI projects, with few accredited health departments selecting that response by 2016. Non-e-PHAB LHDs also reported declines in the percentage reporting no projects in the previous year, from 41.4% in 2010 to 30.6% in 2016. Accordingly, there is an increase in all health departments undertaking some formal QI efforts across the survey periods. Notably, reports that LHDs are doing 7 or more formal projects increased among accredited health departments from 13.4% in 2010 to 22.8% in 2016.
To assess the degree of formal QI implementation within LHDs, the 2013 and 2016 NACCHO surveys asked a series of 8 dichotomous questions regarding the presence of specific elements of a QI program. The percentages of LHDs reporting each of the 8 specific QI-related elements, as well as the percentages that had completed 0 to 2, 3 to 5, and 6 to 8 of them, are presented in Table 1 by accreditation status in June 2017 for the 2013 and 2016 Profile waves (question not asked in 2010). Increases in the number of these QI-related elements indicate an increasingly robust QI structure within the agency. In 2013, approximately one-quarter (26.4%) of accredited LHDs reported conducting between 6 and 8 of those elements; that proportion increased to three-quarters (75.0%) by 2016. Between those survey waves, in-process LHDs increased from 6.8% to 32.2%, while the percentage of non-e-PHAB health departments remained flat. Note that a larger percentage of respondents indicated the presence of formal QI agency-wide than the percentage reporting 6 to 8 QI elements in place. The 1 exception is that the percentages for the 2016 accredited health departments are roughly equivalent.
Examined individually, there is increased uptake across all 8 metrics among accredited LHDs between 2013 and 2016, while the same is true of 7 questions for LHDs in process. Local health departments not in e-PHAB show relatively small changes in uptake (4 positive changes, 4 negative changes). Moreover, the differences between the 2 survey periods are far more pronounced for accredited health departments. An alternative proxy for PM is the use of data to drive improvement efforts. Focusing on that question, large advances were reported among accredited LHDs and those in e-PHAB between 2013 and 2016, while the needle barely moved for LHDs not in e-PHAB. Two elements, “QI is incorporated in employee job descriptions” and “QI is incorporated in employee performance appraisals,” were consistently low across all respondents in both survey years.
Logistic regressions were also conducted in each wave of the survey to control for governance and size of population served (Table 3). In 2010 and 2013, the odds ratios of an accredited LHD having a formal QI program agency-wide, compared with one not in e-PHAB, were 2.9 and 2.7, respectively. However, in 2016, the odds ratio favoring accredited health departments rose to 27.0 (P < .001). For in-process LHDs, the odds ratio was insignificant in both 2010 and 2013 but was 6.0 (P < .001) in 2016. Similar patterns can be observed for the outcomes related to having 6 to 8 elements of the QI program in place and using performance data to drive improvement efforts.
Results from the longitudinal analyses of LHDs that completed the QI module in both 2013 and 2016 are presented in Table 4. Nearly 90% of e-PHAB LHDs reported an increase in the number of the 8 QI elements that were in place. At the same time, approximately one-third of LHDs not in e-PHAB reported increases. Local health departments that were registered in e-PHAB or accredited were significantly (P = .006) more likely than non-e-PHAB LHDs to report any increase in overall level of QI implementation (OR = 4.89), an increase in overall level of QI implementation to the highest level (OR = 5.45), and an increase in the number of the 8 specific QI-related elements in place at their LHD (OR = 16.1).
Accreditation appears to be a strong driver for the uptake of QI and PM among LHDs actively pursuing accreditation, supporting previous evaluation findings.5 The results of this study highlight self-reported activities at different points in time in key domains that seem to consistently indicate a steady march forward in the adoption of milestone QI efforts. Perhaps less appreciated is that accreditation may be floating all boats or, more accurately, many boats. Although penetrance of QI efforts appears of lesser magnitude in non-e-PHAB LHDs, over time they were nonetheless more likely to undertake QI and PM activities, particularly moving from no QI activity to some. For example, about one-third reported an increase in the number of QI elements implemented between 2013 and 2016. These findings about intermediate outcomes, if confirmed, strengthen the case that PHAB is progressing toward achieving its purpose of improving the quality and performance of health departments in the United States.
The case for steady progress among accredited health departments is underscored by notable (sometimes dramatic) increases in the following: adoption of agency-wide formal QI (more than doubling since baseline in 2010); combined agency-wide and specific programmatic formal QI; implementation of QI projects; increase over baseline of those accredited LHDs conducting 4 or more QI projects; profound increases in uptake of the elements contributing to agency-wide QI/PM; and substantial growth in agencies reporting the use of data to drive improvement (1 proxy for assessing PM).
Similar categorical gains were reported by health departments on the pathway toward accreditation, although they lag behind accredited health departments in the 2016 wave. Their responses to the QI/PM questions in 2016 often resembled the responses of the accredited health departments in 2013. This is not unexpected, as many of the LHDs that are in the accredited cohort (based on their accreditation status in 2017) would have been in process when they responded to the 2013 NACCHO Profile. Taken together, these findings suggest that LHDs may see gains in their QI engagement throughout their accreditation journey. Headway toward formal agency-wide QI mirrored accredited health departments, as did implementation of the number of QI projects in most categories.
Local health departments not in e-PHAB also made gains, although more modest, bolstering the evidence that PHAB accreditation benefits the entire public health enterprise. Nonetheless, reported improvements were less robust, widespread, or consistent. More of them had moved toward formal QI implementation compared with the base year survey and more QI projects were being undertaken.
Accredited LHDs reported higher levels of overall QI implementation than in-process or non-e-PHAB LHDs even in 2010, supporting the idea that many of the early adopters of accreditation were already well along the pathway to QI. But the in-process and non-e-PHAB LHDs reported similar levels of overall QI implementation in 2010, and the in-process LHDs showed much more advancement in level of QI implementation over time than the non-e-PHAB LHDs. This suggests that the process of undertaking PHAB accreditation is a catalyst for formalizing and strengthening LHD QI and PM efforts. Furthermore, the greater levels of overall QI implementation and specific QI-related activities in accredited LHDs compared with in process LHDs suggest that LHDs continue to expand their QI efforts after they attain accreditation.
In addition, the role of philanthropic and governmental support of QI/PM early in the development of PHAB accreditation must be acknowledged. Programs such as the Multi-State Learning Collaborative, the Community of Practice for Public Health Improvement, and Gaining Ground, all funded by the Robert Wood Johnson Foundation, helped jumpstart health department QI/PM journeys.16 Similarly, the Centers for Disease Control and Prevention has had an enduring role through nation-wide sponsorship via the National Public Health Improvement Initiative and the ongoing Accreditation Support Initiative benefitting local, state, tribal, and territorial health departments.17
The findings described here are consistent with the theory originally espoused as the rationale for introducing public health accreditation a decade ago.3,18,19 The longitudinal data and impressive odds ratios suggest very strong correlations between accreditation and QI/PM. Carman and Timsina10 also had noted that LHDs intending to seek accreditation were more likely to pursue QI early on. This offers the possibility that PHAB accreditation is both a driver to implement quality within the health department to become accredited and a promoter to advance the maturity of the QI efforts after accreditation has been achieved—and hopefully maintained.
This study offers deeper insight into the impact that accreditation may have on fostering the implementation of critical changes in organizational QI and PM activities. Although this study offers several strengths, including its use of both cross-sectional and longitudinal analyses, large sample size, and consistent metrics, it does have limitations. Selection bias of accredited health departments already likely to have embarked upon substantial improvements in their organizations may portray accredited health departments more favorably. Self-reporting and social response bias may have resulted in more optimistic assessments, although potentially this is true across the entire sample of LHDs. There is also the potential for construct validity challenges. Health department elements of formal agency-wide QI programs assessed in the NACCHO QI module are not perfectly aligned with PHAB Standards and Measures. Similarly, dichotomies between self-assessment of formal QI system status and the more limited implementation of 6 to 8 QI program elements may reflect model construct validity inconsistencies. Finally, this study cannot imply causation, only correlation.
Harnessing the analytic power of combining NACCHO profile and QI module data with data on registration in e-PHAB provides a unique opportunity to examine LHD progress toward implementation of key QI and PM dimensions. Accredited health departments have demonstrated marked development toward QI and PM implementation, as have LHDs registered in e-PHAB, but not yet accredited. To maintain accreditation, health departments have opportunities to build on their QI and PM work, for example, by applying these concepts in collaborations with community partners to address health status.20 Public health as an enterprise is making progress as well, signaling that accreditation can be a powerful driver to improve health department performance. Ongoing research and additional longitudinal studies will be necessary to assess overall system growth and advancement to determine whether this evolution progresses and ultimately leads to health status advancement.
Implications for Policy & Practice
- Self-reported engagement in QI/PM is greater among PHAB-accredited health departments than those health departments that are not pursuing accreditation. As health departments are contemplating whether to apply for accreditation, this potential for developing a more robust QI/PM system—and the increases in efficiency and effectiveness that may accompany this—should be taken into account.
- Gains in QI/PM are experienced while the health departments are in process to achieve accreditation. While preparing for accreditation and maintaining their accreditation status, health departments may want to focus on how to bolster their QI/PM work, moving from ad hoc QI projects to ensuring that it is more formally incorporated in their culture.
- The milieu supporting greater LHD engagement in QI activities has resulted in gains for many health departments—whether or not PHAB accreditation has been pursued.
- Further research that compares health departments in their QI/PM work, as well as in other dimensions of their performance, based on accreditation status, may help the field better understand the impact of accreditation.
1. Beitsch LM, Corso LC, Davis MV, Joly BM, Kronstadt J, Riley WJ. Transforming public health practice through accreditation
(a user guide for the special accreditation
issue). J Public Health Manag Pract. 2014;20(1):2–3.
2. Beitsch LM, Riley W, Bender K, Embedding quality improvement
: evolving from theory to practice. J Public Health Manag Pract, 2014; 20(1):57–60.
3. Bender K, Beitsch L, Kronstadt J, Development of a national public health accreditation
program: a case study in strategic change management and sustainability. J Public Health Manag Pract, 2017;23(3):315–324.
5. Kronstadt J, Meit M, Siegfried A, Nicolaus T, Bender K, Corso L. Evaluating the impact of National Public Health Department Accreditation
—United States, 2016. MMWR Morb Mortal Wkly Rep. 2016;65(31):803–806.
6. Davis MV, Mahanna E, Joly B, et al Creating quality improvement
culture in public health agencies. Am J Public Health. 2014;104(1):e98–e104.
7. Hill A, Wolf HJ, Scallan E, Case J, Kellar-Guenther Y. Drivers motivating community health improvement plan completion by local public health agencies and community partners in the rocky mountain region and western plains. Supplement, Community Health Assessment. J Public Health Manag Pract, 2017;23 (suppl 4):S39–S46.
8. Singh SR, Carlton EL. Exploring the link between completion of accreditation
prerequisites and local health departments' decision to collaborate with tax-exempt hospitals around the community health assessment. J Public Health Manag Pract. 2017;23(2):138–147.
9. Wilson KD, Mohr LB, Beatty KE, Ciecior A. Describing the continuum of collaboration among local health departments with hospitals around the community health assessments. J Public Health Manag Pract. 2014;20(6):617–625.
10. Carman AL, Timsina L. Public health accreditation
: rubber stamp or roadmap for improvement. Am J Public Health. 2015;105(suppl 2):S353–S359.
11. Public Health Accreditation
Board. Standards & Measures Version 1.5. Alexandria, VA: Public Health Accreditation
12. National Association of County & City Health Officials. 2010 National Profile of Local Health Departments. Washington, DC: National Association of County & City Health Officials; 2011.
13. National Association of County & City Health Officials National Profile of Local Health Departments. Washington, DC: National Association of County & City Health Officials; 2014.
14. National Association of County & City Health Officials. 2016 National Profile of Local Health Departments. Washington, DC: National Association of County & City Health Officials; 2017.
15. Public Health Accreditation
Board. e-PHAB Health Department
List. Alexandria, VA: Public Health Accreditation
16. Beitsch LM, Thielen L, Mays G, et al The multistate learning collaborative, states as laboratories: informing the national public health accreditation
dialogue. J Public Health Manag Pract. 2006;12(3):217–231.
17. McLees AW, Thomas CW, Nawaz S, Young AC, Rider N, Davis M. Advances in public health accreditation
readiness and quality improvement
: evaluation findings from the National Public Health Improvement Initiative. J Public Health Manag Pract, 2014;20(1):29–35.
18. Beitsch LM, Yeager VA, Moran J. Deciphering the imperative: translating public health quality improvement
into organizational performance management
gains. Annu Rev Public Health. 2015;36:273–287.
19. Bender K, Moehrle C. PHAB
: reflections from the first year of accreditation
. J Public Health Manag Pract. 2014;20(1):7–8.
20. Chapman RW. Public health accreditation
impact: the performance management
system. Supplement, Impact of Public Health Accreditation
. J Public Health Manag Pract. 2018;24(suppl 3):S19–S21.