Share this article on:

Changes in Local Public Health System Performance Before and After Attainment of National Accreditation Standards

Ingram, Richard, C., DrPH; Mays, Glen, P., PhD; Kussainov, Nurlan, MPA

Journal of Public Health Management and Practice: May/June 2018 - Volume 24 - Issue - p S25–S34
doi: 10.1097/PHH.0000000000000766
PARTNERSHIPS: Research Reports: Research Full Report

Objective: The aim of this study is to investigate the impact of Public Health Accreditation Board (PHAB) accreditation on the delivery of public health services and on participation from other sectors in the delivery of public health services in local public health systems.

Design: This study uses a longitudinal repeated measures design to identify differences between a cohort of public health systems containing PHAB-accredited local health departments and a cohort of public health systems containing unaccredited local health departments. It uses data spanning from 2006 to 2016.

Setting: This study examines a cohort of local public health systems that serves large populations and contains unaccredited and PHAB-accredited local health departments.

Participants: Data in this study were collected from the directors of health departments that include local public health systems followed in the National Longitudinal Study of Public Health Systems.

Intervention: The intervention examined is PHAB accreditation.

Main Outcome Measures: The study focuses on 4 areas: the delivery of core public health services, local health department contribution toward these services, participation in the delivery of these services by other members of the public health system, and public health system makeup.

Results: Prior to the advent of accreditation, public health systems containing local health departments that were later accredited by PHAB appear quite similar to their unaccredited peers. Substantial differences between the 2 cohorts appear to manifest themselves after the advent of accreditation. Specifically, the accredited cohort seems to offer a broader array of public health services, involve more partners in the delivery of those services, and enjoy a higher percentage of comprehensive public health systems.

Conclusions: The results of this study suggest that accreditation may yield significant benefits and may help public health systems develop the public health system capital necessary to protect and promote the public's health.

University of Kentucky College of Public Health, Department of Health Management and Policy, Lexington, Kentucky.

Correspondence: Richard C. Ingram, DrPH, University of Kentucky College of Public Health, 111 Washington Ave, Lexington, KY 40536 (rcingr2@uky.edu).

Funding for some of this research was provided by the Robert Wood Johnson Foundation and the United States Centers for Disease Control and Prevention.

The authors declare no conflicts of interest.

The voluntary national accreditation program supported by the Public Health Accreditation Board (PHAB), since its inception in 2011, has sought to transform public health practice in the United States by measuring performance against a series of consensus-based, nationally recognized standards.1 An emerging body of evidence supports the expectation that participation in the PHAB accreditation process helps state and local public health agencies achieve higher levels of community engagement2–5 and greater focus on core organizational and community activities like a community health assessment, community health improvement planning, and organizational strategic planning.6 Because public health practice is a multiorganizational and multisectoral enterprise, there is a growing interest in understanding how PHAB accreditation influences the entirety of the public health delivery system. If PHAB accreditation is successful in influencing practices that extend beyond the organizational boundaries of governmental public health agencies, then it may serve as an important long-term driver of improved population health and well-being.

Identifying the system-level impact of PHAB accreditation on public health practices that require the collective actions of multiple sectors and community organizations is a research priority for PHAB.7 Investigating the influence of PHAB accreditation on these areas is particularly important, given the pressures facing local health departments in the United States. Many local health departments operate in a climate characterized by declining or flat funding and resultant program cuts.8–10 Efforts to protect public health in the face of declining revenues may necessitate refocusing on the delivery of core public health services9–11 and promoting greater involvement in public health activities by other members of the public health system.12–14

The PHAB accreditation process holds particular promise as a means to protect the delivery of public health services, given PHAB's focus on core public health activities and on encouraging the development of relationships with a wide variety of community partners. Evidence suggests that local public health systems that offer a broad scope of population-based preventive services, with the local health department playing a central role in the delivery of those services, and that involve partners from a diverse and dense array of sectors in the delivery of those services, enjoy superior health outcomes when compared their peers that offer fewer services, and that engage partners from fewer sectors. They enjoy significantly lower rates of all-cause mortality, as well as mortality related to heart disease, diabetes, cancer, and influenza. Systems that display these types of public health system capital (a broad scope of services, the local health department playing a central role in the delivery of those services, and engagement of a dense array of public health system partnerships) are known as comprehensive public health systems.15

Certain requirements of PHAB accreditation could aid in the development of the types of public health system capital necessary to develop comprehensive public health systems. The Public Health Accreditation Board requires accredited health departments to have used a collaborative process to complete a community health assessment, with engagement of a wide variety of community representatives, and evidence suggests that public health agencies that complete a community health assessment may experience noticeable increases in community participation.2 , 3 , 5 The Public Health Accreditation Board also requires accredited departments to have completed and implemented a community health improvement plan, again, with involvement of a wide variety of community representatives. Increases in community participation in health department activities could help health departments build stronger relationships with other community organizations. This may encourage greater engagement from members of the public health system, outside local government, in delivering public health services. Public Health Accreditation Board accreditation is also focused exclusively on public health services and not individual or clinical services.16 As a result, PHAB accreditation could result in departments shifting their focus away from delivering personal health services and toward delivering a wider array of public health activities.

The aim of this study is to investigate the relationship between PHAB accreditation and the development of public health system capital: specifically, on the delivery of public health services, on participation from other sectors in the delivery of public health services, and on the development of comprehensive public health systems. It uses a longitudinal repeated measures design to identify differences between a cohort of public health systems containing PHAB-accredited local health departments and a cohort of public health systems containing unaccredited local health departments. The study focuses on 4 areas: the delivery of core public health services, local health department contribution toward these services, participation in the delivery of these services by other members of the public health system, and the development of comprehensive public health systems.

Back to Top | Article Outline

Methods

Data from the National Longitudinal Survey of Public Health Systems (NLSPHS) were used to calculate availability of public health activities, local health department contribution to these activities, and contribution from other sectors to these activities.17 This survey tracks a nationally representative cohort of US communities over time, beginning originally in 1998 and continuing through 2016. The survey is sent to the chief executive of the local public health department in each system and is filled out by the chief executive or its designee. The NLSPHS uses a set of validated survey questions developed to measure the performance of 20 activities, contained in Table 1, related to the 3 core functions of public health.18 , 19 The activities measured by the survey reflect strategies that have been widely recommended by national professional associations, federal agencies, and expert consensus panels for implementation in every community to protect and improve population health. The NLSPHS asks questions focused on 4 domains for each activity: availability (whether the activity is implemented by the public health system), governmental public health effort (the % effort, using a 5-point Likert scale, the local health department contributes to the activity), perceived effectiveness of each activity (using a 5-point Likert Scale), and multisectoral contributions (which other sectors of the public health system, displayed in Table 5, contribute to activities).

TABLE 1

TABLE 1

The NLSPHS has followed a cohort of local public health systems that serve large (100 000 residents or larger) populations since 1998. The survey has been administered to the same local health departments 5 times (1998, 2006, 2012, 2014, and 2016) via the US mail and electronic surveys. Data from the 2006-2016 surveys were used for this study to provide a complete view of public health system attributes for 2 periods before, and 2 periods after, PHAB accreditation (while PHAB accreditation was launched in 2011, the first local health departments were not accredited by PHAB until February 2013, after the 2012 survey).

The NLSPHS data were separated into 2 cohorts—those from local public health systems containing PHAB-accredited health departments as of June 2017, and those from local public health systems not containing PHAB-accredited health departments. Data from the 2 cohorts were kept separate for all 4-time periods of the survey—regardless of when PHAB accreditation was attained. So, while a local health department may not have attained PHAB accreditation until 2014, data from that department were included in the PHAB-accredited cohort from 2006 onward. This allows the authors to identify differences that may have existed between the 2 cohorts before PHAB accreditation was attained. It was not possible for a system to move out of the PHAB-accredited cohort—any department accredited during the time period of the study would still be accredited at the study's end, in 2016.

The availability of each of the 20 NLSPHS activities was determined by calculating the percentage of respondents in each cohort who reported that service was offered in the jurisdiction covered by their local public health system. An equality of proportions test was performed to identify any significant differences between the percentage of the PHAB-accredited and unaccredited cohort offering each activity for each survey period (2006, 2012, 2014, and 2016). The mean departmental effort contributed to each activity was determined for each cohort by calculating the average percentage of effort reported by respondents in each cohort for each activity. A t test was performed to identify any significant differences between the departmental effort reported for each activity in the PHAB-accredited and unaccredited cohort for each survey period. Mean percent contribution from each sector to public health activities was determined by calculating the average number of activities contributed to by each sector, as reported by each local health department that responded to the survey. A t test was performed to identify any significant differences between contribution from each sector by the PHAB-accredited and unaccredited cohort for each survey period. Public health systems in each cohort were classified as comprehensive or not comprehensive using results from hierarchical cluster analysis, described in a previous publication,17 that examined variations in the scope of activities offered in each public health system, the density of contribution from other public health system partners in the delivery of these activities, and the degree to which the local health department was central to the delivery of these activities. The percentage of comprehensive public health systems in each cohort was then calculated. An equality of proportions test was used to identify any significant differences that existed between the 2 cohorts. All analysis was performed using STATA version 14.

Back to Top | Article Outline

Results

Table 2 contains the number and percentage of public health systems from 2006 to 2016 containing PHAB-accredited and unaccredited local health departments in our sample of the NLSPHS. The number of respondents ranged from 236 to 331. While the number of systems containing PHAB-accredited and unaccredited departments varied between 2006 and 2016, the percentage of systems containing PHAB-accredited and unaccredited departments was relatively stable (between 19% and 22% and 81% and 78%, respectively).

TABLE 2

TABLE 2

Public health systems containing PHAB-accredited local health departments differ markedly from their unaccredited peers, and this seems to manifest itself to a large degree after PHAB's accreditation program launched in 2011. The 2006 and 2012 NLSPHS results reflect marginal differences between cohorts in a vast majority of measures. By 2016, the PHAB-accredited cohort tended to offer a higher percentage of public health activities, contribute more effort to almost all of those activities, and enjoy higher levels of contribution from most other public health system partners to public health activities. This is particularly notable, given that the unaccredited cohort offered higher levels of availability of, and contributed more effort to, many public health services when compared with PHAB-accredited cohort for the 2006-2012 time period.

Back to Top | Article Outline

Availability

Table 3 contains the percentage of public health systems in the PHAB-accredited and unaccredited cohorts that report offering each NLSPHS activity from 2006 to 2016. The unaccredited cohort reflects lower overall levels of delivery of public health activities over time, with decreases seen in 11 of the 20 activities from 2006 to 2016. Decreases were concentrated in the activities associated with assurance, with 5 of the 7 activities reflecting lower levels of availability. In contrast, the PHAB-accredited cohort seems to reflect greater overall levels of delivery of public health activities over time, with increases in availability of 14 of 20 activities from 2006 to 2016. Large increases, ranging from 17% to 44%, were concentrated in 5 of the 7 activities associated with policy development.

TABLE 3

TABLE 3

Significant differences between the accredited and unaccredited cohort existed for only 2 activities in 2006 and 3 activities in 2012. Substantial change began to manifest itself in the 2014 survey—in 2014, the PHAB-accredited cohort displayed statistically significantly higher levels of availability than the unaccredited cohort in 9 of the 20 activities examined. In 2016, significantly higher differences were observed in the PHAB-accredited cohort for 9 of the 20 activities as well. Significant differences between both groups in 2014 and 2016 were concentrated in 6 of the 7 activities associated with policy development. The unaccredited cohort offered a greater availability than the PHAB-accredited cohort in 8 of the 20 activities in 2006, and 6 in 2012. In sharp contrast to 2006 and 2012, the unaccredited cohort reported greater availability for only 1 activity in 2014 and did not offer greater availability than the PHAB-accredited cohort for any activity in 2016.

Back to Top | Article Outline

Departmental contribution

Table 4 contains the percentage of effort contributed by local public health departments to NLSPHS activities from 2006 to 2016. The PHAB-accredited cohort reflects increased departmental contribution to public health activities over time. Departmental contribution to public health activities increased in 13 of 19 activities from 2006 to 2016 in the PHAB-accredited cohort. Six activities displayed increases of 10% or higher. The unaccredited cohort reflected little change in departmental contribution to most public health activities from 2006 to 2016—while increases were observed in 10 activities, most changes observed were less than 5%.

TABLE 4

TABLE 4

Statistically significant differences between the accredited and unaccredited cohort exist for only 1 activity in 2006 and 3 activities in 2012. Substantial change began to manifest itself in the 2014 survey—in 2014, the PHAB-accredited cohort reported significantly higher levels of effort than the unaccredited cohort in 12 of the 19 activities examined. In 2016, the PHAB-accredited cohort reported significantly higher levels of effort in 6 of the 19 NLSPHS activities. Significant differences between both groups in 2014 were concentrated in 6 of the 7 activities associated with policy development; significant differences in 2016 were found in 3 of the 7 activities associated with policy development.

Back to Top | Article Outline

Multisectoral contribution to population health activities

Table 5 displays the average percent contribution from other sectors to public health activities in the unaccredited and PHAB-accredited cohorts from 2006 to 2016. The unaccredited cohort reflects a trend of lower levels of involvement in public health activities by other public health system partners. Decreased involvement, ranging from −2% to −17%, was observed in 7 of the 12 sectors examined. The PHAB-accredited cohort reflects a trend of modestly higher levels of involvement in public health activities by other public health system partners. Decreased involvement was observed from local, state, and federal government entities in both cohorts.

TABLE 5

TABLE 5

Significant differences between the PHAB-accredited and unaccredited cohorts in contribution to NLSPHS activities were found in only 2 of the 12 sectors in 2006 and none of the 12 sectors in 2012. In contrast, the PHAB-accredited cohort reported significantly higher levels of involvement in 7 of the 12 sectors in 2012 and 6 of the 12 sectors in 2016. The unaccredited cohort reported significantly higher levels of involvement than the PHAB-accredited cohort from 1 sector, physician practices, in 2016.

Back to Top | Article Outline

Public health system capital

The Figure displays the percentage of comprehensive public health systems in each cohort. Slight decreases are observed in both cohorts between 2006 and 2012. Increases are observed for both cohorts for 2014 and 2016, but the increases in the accredited cohort are much larger in magnitude. Throughout the study period, the accredited cohort displays a higher proportion of comprehensive systems than the unaccredited cohort. However, significant differences are observed only for the 2014 and 2016 cohorts, where the accredited cohort has a higher percentage of comprehensive public health systems.

FIGURE

FIGURE

Back to Top | Article Outline

Discussion

The significant differences in availability of NLSPHS activities between the PHAB-accredited and unaccredited cohort in 2014 and 2016 lend support to the idea that the accreditation process may lead to positive changes in public health delivery system capital, given the relative similarity of the cohorts in 2006 and 2012, and that many of the changes take place after the health departments in the PHAB-accredited cohort could have engaged in the accreditation process. The notion that accreditation may lead to positive system–level changes is further supported by the fact that the unaccredited cohort offers greater availability of many activities prior to the launch of accreditation. This, combined with the marginal differences between cohorts in 2006 and 2012, suggests that accreditation may be associated with these differences.

Significant differences in the availability of NLSPHS activities offered between the unaccredited and PHAB-accredited cohorts seem to manifest themselves to a particularly large degree in the NLSPHS activities associated with policy development. Public Health Accreditation Board accreditation places a substantial focus on local health department engagement with governing entities, and policy makers writ large, so this may be expected in the PHAB-accredited cohort. It is important to note, however, that the NLSPHS measures activities related to the public health system, not just the local health department. Thus, our results suggest that accreditation may lead to measurable change in not just health department activities but also policy development–related activities involving the larger public health system.

The PHAB-accredited cohort also displayed significantly greater levels of effort in the delivery of public health services than the unaccredited group for many activities in 2014 and, to a lesser degree, in 2016. Differences between the 2 groups in 2006 and 2012 were negligible at best for most activities. This, combined with the changes observed in availability, supports the idea that accreditation may be associated with positive changes in the delivery of public health services.

The significant differences in involvement in NLSPHS activities by public health system members from other sectors between the PHAB-accredited and unaccredited cohort in 2014 and 2016 also lend support to the idea that the accreditation process may lead to positive changes in public health delivery system capital, given the relative similarity of the cohorts in 2006 and 2012. The PHAB-accredited cohort appears to engage both governmental and nongovernmental public health system partners from a broad array of sectors.

It may not be surprising that, given the significantly higher levels of availability of many public health service, departmental effort to many of those services, and multisector engagement to public health activities displayed in the PHAB accredited cohort, the PHAB-accredited cohort also contained a significantly higher percentage of comprehensive public health systems than the unaccredited cohort in 2014 and 2016, after the advent of accreditation. Given the strong evidence that comprehensive systems are associated with a host of positive health outcomes, this could provide early evidence that the accreditation process could be associated with positive changes in health status.

One problem that confounds attempts to identify the impact of accreditation on health department characteristics and outcomes is selection bias—some suggest that higher-performing health departments will be more likely to pursue accreditation.20 Thus, any differences associated with unaccredited and PHAB-accredited departments will be attributable to preexisting conditions, not the accreditation process. Our results suggest that this is not the case and that the PHAB-accredited cohort is similar to (and in some measures scores lower than) the unaccredited cohort in the preaccreditation period. The similarities between the unaccredited and PHAB-accredited cohort in 2006 and 2012 also suggested that the marginal benefit of PHAB accreditation may be larger for communities represented by PHAB-accredited health departments than previously thought.

It is important to note that this study does have limitations. Because of its design, it does not imply causation and only suggests that correlation may exist. Further study is needed to determine the impact of PHAB accreditation on public health system makeup and activities. It relies on self-reported data provided by the chief executive, or its designee, of the local public health agency in each community examined. As a result, it is subject to commonly discussed sources of measurement error associated with self-reported data, such as social desirability bias. Given the length of time it takes local health departments to complete the PHAB-accreditation process, it is possible that some systems represented by departments in the unaccredited cohort were pursuing accreditation during the time period covered—this may help mask differences between the 2 cohorts. While the NLSPHS focuses on 20 activities derived from the 3 core functions of public health, it may not encompass all the public health activities offered in local public health systems. It is also important to note that, while the NLSPHS is grounded in the 3 core functions of public health, and the NLSPHS does align with many PHAB standards and measures, the development and use of instruments designed to dovetail more closely with PHAB's Standards and Measures may provide an even broader picture of the impact of accreditation on local public health system makeup.

However, the study also has some notable strengths. The activities contained in the NLSPHS are derived from the core functions of public health. As a result, they cover a wide breadth of assurance, assessment, and policy-development relayed activities offered by local public health systems. Unlike many instruments used to examine activities and characteristics of public health agencies in the United States, the NLSPHS instrument has been validated.19 This suggests that the changes reflected in the survey are not a result of measurement error.

This study provides early evidence of the impact PHAB accreditation has on public health systems. Public Health Accreditation Board–accredited systems seem to enjoy improvements in service delivery, as measured by the array of public health activities offered; seem to display increased focus on the delivery of public health services, as measured by the percent effort the local health department contributes to these activities; and seem to do this with a greater array of partners, when compared with their unaccredited peers. This may lead to positive improvements in health status—evidence suggests that these communities with public health systems that offer a broad array of services and involve partners from other sectors enjoy improved health status and are more likely to implement evidence-based approaches to disease prevention such as comprehensive smoke-free laws.15 This suggests that, while accreditation comes with significant costs, it also yields significant benefits and may help protect and promote the public's health.

Back to Top | Article Outline

Implications for Policy & Practice

  • Little evidence exists that examines the impact of PHAB accreditation on public health department or public health system attributes or activities.
  • Given PHAB's mission to transform public health practice in the United States and improve the quality of services delivered by public health agencies, identifying the impact of accreditation on public health practice is an essential component of any attempts to evaluate the benefits of accreditation and to ensure that PHAB accreditation is an effective quality improvement and assurance mechanism.
  • The results of this study provide early evidence that supports the notion that accreditation is an effective means to improve the quality of the delivery of public health services by local public health systems and protect and improve public health.
Back to Top | Article Outline

References

1. Ingram RC, Bender K, Wilcox R, Kronstadt J. A consensus-based approach to national public health accreditation. J Public Health Manag Pract. 2014;20:9–13.
2. D'Amore D, Bretherton J. Norwalk Health Department: coleading a community health assessment and improvement plan with Norwalk hospital. J Public Health Manag Pract. 2014;20:73–75.
3. Kronstadt J, Meit M, Siegfried A, Nicolaus T, Bender K, Corso L. Evaluating the impact of National Public Health Department Accreditation—United States, 2016. MMWR. 2016;65:803–806.
4. McLees AW, Thomas CW, Nawaz S, Young AC, Rider N, Davis M. Advances in public health accreditation readiness and quality improvement: evaluation findings from the National Public Health Improvement Initiative. J Public Health Manag Pract. 2014;20:29–35.
5. Riccardo J, Parent C, DeSalvo K. New Orleans Health Department: using the accreditation framework to transform a local health department. J Public Health Manag Pract. 2014;20:66–69.
6. Beatty KE, Mayer J, Elliott M, Brownson RC, Wojciehowski K. Patterns and predictors of local health department accreditation in Missouri. J Public Health Manag Pract. 2015;21:116–125.
7. Kronstadt J, Beitsch LM, Bender K. Marshaling the evidence: the prioritized public health accreditation research agenda. Am J Public Health. 2015;105(suppl 2):S153–S158.
8. Trust for America's Health. Investing in America's Health: A State-by-State Look at Public Health Funding and Key Health Facts. Washington, DC: Trust for America's Health; 2013.
9. Willard R, Shah GH, Leep C, Ku L. Impact of the 2008-2010 economic recession on local health departments. J Public Health Manag Pract. 2012;18:106–114.
10. Ye J, Leep C, Newman S. Reductions of budgets, staffing, and programs among local health departments: results from NACCHO's economic surveillance surveys, 2009-2013. J Public Health Manag Pract. 2015;21:126–133.
11. Mays GP, Hogg RA. Economic shocks and public health protections in US metropolitan areas. Am J Public Health. 2015;105(suppl 2):S280–S287.
12. Johnston LM, Finegood DT. Cross-sector partnerships and public health: challenges and opportunities for addressing obesity and noncommunicable diseases through engagement with the private sector. Ann Rev Public Health. 2015;36:255–271.
13. Mays GP, Scutchfield FD. Improving public health system performance through multiorganizational partnerships. Prev Chronic Dis. 2010;7:A116.
14. Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Ann Rev Public Health. 2000;21:369–402.
15. Mays GP, Mamaril CB, Timsina LR. Preventable death rates fell where communities expanded population health activities through multisector networks. Health Aff. 2016;35:2005–2013.
16. Public Health Accreditation Board. Public Health Accreditation Board Standards and Measures Version 1.5. Alexandria, VA: Public Health Accreditation Board; 2013.
17. Mays GP, Scutchfield FD, Bhandari MW, Smith SA. Understanding the organization of public health delivery systems: an empirical typology. Milbank Q. 2010;88:81–111.
18. Turnock BJ, Handler AS, Miller CA. Core function-related local public health practice effectiveness. J Public Health Manag Pract. 1998;4:26–32.
19. Miller CA, Richards TB, Davis SM, et al Validation of a screening survey to assess local public health performance. J Public Health Manag Pract. 1995;1:63–71.
20. Joly BM, Polyak G, Davis MV, et al Linking accreditation and public health outcomes: a logic model approach. J Public Health Manag Pract. 2007;13:349–356.
Keywords:

accreditation; performance measurement; quality improvement

Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved.