The national public health accreditation program, which was launched by the Public Health Accreditation Board (PHAB) in September 2011, establishes a set of standards that public health departments must achieve to be recognized as a nationally accredited health department.1 Since its implementation, there has been a steady increase in the percentage of local health departments (LHDs) formally involved in PHAB accreditation. Formal involvement in accreditation begins when a health department registers its intent to apply for accreditation in PHAB's electronic information system, e-PHAB. According to the 2016 National Association of County & City Health Officials (NACCHO) Profile of Local Health Departments (Profile survey), 21% of all LHDs were formally involved in PHAB accreditation. In 2016, an additional 18% of LHDs reported that they planned to apply for accreditation, 31% of LHDs were undecided on whether they would apply, and 20% had decided not to apply.2 Factors that may influence an LHD's decision to pursue accreditation include the time and effort required to achieve accreditation, the fees for undergoing the accreditation process, and the PHAB accreditation standards exceeding the LHD's capacity.2–4 Large LHDs that serve a population greater than 500 000 were the most likely to report formal involvement in accreditation (58%) compared with smaller LHDs serving fewer than 50 000 individuals (12%).2 Research has shown that LHDs in states where the state health department applied for accreditation were more likely to be favorably inclined toward accreditation.4
The initial effects and benefits of PHAB accreditation have begun to be documented.5,6 To understand how the accreditation program has impacted health departments that are not formally involved in the accreditation process, NORC at the University of Chicago conducted surveys of health departments that had achieved PHAB accreditation as well as those that had not yet applied for PHAB accreditation. These surveys were implemented with funding from (1) the Robert Wood Johnson Foundation (RWJF),7,8 and (2) PHAB, through funding provided by RWJF.9 This article describes findings from 2 surveys: one of health departments that were not yet formally involved in the PHAB accreditation program, and one of health departments that had been accredited for 1 year.
We implemented one Web-based survey of health departments that had no formal involvement in the accreditation program (referred to as the nonapplicant survey). We also implemented one Web-based survey of accredited health departments 1 year after they had achieved accreditation (referred to as the postaccreditation survey). The goal of these surveys was to assess the following: (1) how the accreditation program has affected nonapplicant health departments; (2) differences in quality improvement (QI) infrastructure between nonapplicant health departments and accredited health departments; and (3) barriers and challenges to pursuing accreditation. In the following text, we describe the sampling techniques, data collection procedures, and analysis methods for the nonapplicant and postaccreditation surveys. Both surveys were completed by health officials or their designee. The NORC institutional review board reviewed the survey protocols and determined the study to be exempt from full review.
In spring 2016, NORC developed a Web-based survey instrument for health departments that were not formally involved in accreditation. The nonapplicant survey instrument included open- and closed-ended questions regarding the following: plans to apply for accreditation; barriers to pursuing accreditation; completion of a community health assessment (CHA), a community health improvement plan (CHIP), and a health department strategic plan; and perceived benefits of accreditation such as strengthened QI activities. The CHA, the CHIP, and the strategic plan are 3 documents that must be in place when a health department applies for PHAB accreditation. PHAB definitions for these documents were included in the survey.10
The nonapplicant survey was fielded 2 times: once in mid-2016 and again in mid-2017. The survey instrument used in 2017 was the same as the instrument used in 2016, with the exception of several new questions regarding the perceived benefits of PHAB accreditation included in the 2017 instrument. The 2016 survey was fielded to local, state, and tribal health departments. The 2017 survey was fielded only to LHDs.
In 2016, we contacted the universe of state health departments that were not formally involved in PHAB accreditation. To identify the state health departments to contact, NORC compiled a list of all state health departments and removed those that were either already accredited or in the process of applying (ie, had registered in e-PHAB). The names of current applicants and accredited health departments were identified by reviewing PHAB data. We identified 18 state health departments that had not yet applied for PHAB accreditation as of June 2016. Of these 18 state health departments, 8 completed the survey.
To identify the tribal health department sample, NORC compiled a list of tribes that were known to have considered, planned for, or begun working on PHAB accreditation activities. NORC confirmed that these tribes had not yet registered in e-PHAB by reviewing PHAB data. We identified 21 tribal health departments that had not yet applied for PHAB accreditation as of June 2016. Of the 21 tribes in the original sample, 1 completed the survey.
The LHD sample was developed by NACCHO in 2016 and 2017. Both years, NACCHO developed a random, stratified sample of 200 LHDs that had not yet applied for accreditation based on their responses to 1 question in the 2016 NACCHO Profile survey.2 LHDs were asked the following: “Which of the following best describes your LHD's participation in the PHAB national accreditation program for LHDs?” LHDs that responded to the question by indicating that they were already accredited or had already registered in e-PHAB were excluded from the sampling frame. For the purposes of the survey, LHDs that were not applying for accreditation were defined as those that responded to the NACCHO Profile survey by indicating that they planned to apply for PHAB accreditation but had not yet registered in e-PHAB, had not decided about whether to apply for PHAB accreditation, or had decided not to apply for PHAB accreditation. NACCHO also excluded LHDs serving jurisdictions with fewer than 10 000 individuals. In addition, for the 2017 sample, NACCHO excluded LHDs that were included in the 2016 sample.
From this sampling frame, NACCHO selected 200 LHDs each year. The sample was stratified by size of population served (<50 000; 50 000-499 999; and >500 000) and geographic region (Northeast, Midwest, South, and West). LHDs with large population sizes were oversampled, since they represent a relatively small portion of all LHDs, to ensure a sufficient number for analysis. NACCHO identified alternate LHDs to be used if they did not have contact information for the health department's director. In 2016, 61 LHDs completed the survey, and in 2017, 80 completed the survey. See Table 1.
To field the survey, NORC sent an invitation via e-mail. For the state sample, we identified each health department's director and contact information. For the tribal sample, we identified the health department director or, when that information was not available, an individual with a leadership role in health-related efforts to serve as the point of contact for the survey. For the LHD samples, NACCHO provided contact information for each health department director. The survey was in the field for approximately 6 weeks, and we sent 3 reminders to complete the survey. Overall, 150 respondents completed the survey in 2016 and 2017; the response rate was 34%.
Postaccreditation health department survey
NORC developed a survey for accredited health departments that reached their 1-year anniversary of achieving accreditation. The survey instrument included questions about the PHAB accreditation process and the perceived benefits 1 year following the accreditation decision, including questions related to QI and performance management. We used a total population sample to survey the universe of health departments that reached their 1-year anniversary of achieving accreditation between December 2015 and May 2017. We fielded the survey to 6 cohorts of health departments as they reached this milestone. Overall, 57 respondents completed the survey, and the response rate was 89%.
When each of the accredited health departments reached the 1-year milestone, PHAB provided a list of the health department names and contact information for the director and the designated accreditation coordinator. This process ensured that we contacted the universe of accredited health departments. We sent an e-mail invitation to health department directors that contained a unique link to the survey. We also sent an e-mail to accreditation coordinators, informing them that we had contacted their health department's director.
For both surveys, NORC researchers reviewed and cleaned the raw data, including recoding response options and corresponding variables as appropriate. For the purposes of data analysis, we combined the responses received from the 2016 and 2017 nonapplicant surveys. Quantitative analyses included descriptive statistics for the nonapplicant survey. To understand how health departments that reportedly plan to apply for accreditation differ from those that are undecided or have decided not to apply, responses to survey questions were compared for these 3 groups of nonapplicants. In addition, for questions related to QI knowledge and infrastructure that were fielded in both the nonapplicant and postaccreditation surveys, we compared the responses of nonapplicants and health departments that had been accredited for 1 year to identify any differences. Finally, for the qualitative data collected in the nonapplicant survey, we used an inductive coding process to code responses to open-ended questions into thematic categories.
Awareness of PHAB among nonapplicants
Public health departments that responded to the survey recognize PHAB and the accreditation program. Among the nonapplicant survey respondents, 98% reported that they were aware of PHAB and the accreditation program. Among those that were aware of PHAB, 39% reported that they had future plans to apply for accreditation, 22% said they had decided not to apply, and 39% were undecided about applying.
Nonapplicants conducting the CHA, CHIP, and strategic plans
When applying for PHAB accreditation, health departments must have completed a CHA, a CHIP, and a strategic plan.10 The majority of non-applicant respondents reported that they had conducted or were currently conducting these 3 documents in the last 5 years: CHA (89%); CHIP (76%); and strategic plan (81%). Figure 1 presents this data stratified by participation type: planning to apply, undecided about applying, or not applying. Notably, 84% of respondents that planned to apply for accreditation had conducted or were currently conducting a CHIP in the past 5 years compared with 63% of nonapplicants that did not plan to apply for accreditation.
Nonapplicants referencing PHAB requirements for the CHA, CHIP, and strategic plan
Among all nonapplicant respondents that reported conducting the CHA, CHIP, and strategic plan, 61%, 63%, and 58%, respectively, had referenced PHAB requirements for these documents. Among nonapplicants that planned to apply for accreditation, at least 80% of respondents reported that they were referencing PHAB requirements. Approximately one-half of nonapplicants that were undecided about applying reported that they were referring to PHAB requirements to develop their CHA, CHIP, and strategic plan. Even among nonapplicant health departments that had decided they would not apply for accreditation, more than a quarter of respondents indicated that they referenced PHAB requirements. Figure 2 presents the percentage of nonapplicant health departments, by participation status, that were reportedly referencing PHAB requirements when developing their CHA, CHIP, and strategic plan.
Perceived benefits of accreditation among nonapplicants
Nonapplicants were asked whether their health department had experienced any changes or outcomes as a result of the PHAB accreditation program's impact on the public health field. These perceived benefits are listed in Table 2, which shows the percentage of nonapplicant survey respondents that strongly agreed or agreed with several statements regarding the impact of the accreditation program. Nonapplicants that reportedly plan to apply for accreditation indicated that the PHAB accreditation program had helped their health department become more aware of their strengths and weaknesses (91%), stimulated QI and performance improvement opportunities within their health department (77%), and increased awareness of or focus on QI efforts within their health department (74%). Approximately one-half of health departments that were reportedly undecided about applying for accreditation indicated that the accreditation program had helped their health department become more aware of their strengths and weaknesses (55%) and increased awareness of or focus on QI efforts within their health department (47%). In addition, 30% of health departments reportedly not applying for accreditation said the accreditation program had increased awareness of QI. Overall, health departments that reportedly plan to apply for accreditation were the most likely to report benefits their health department had experienced as a result of the accreditation program's impact on the field of public health.
At the end of the survey, nonapplicants were asked whether their health department had experienced other benefits (directly or indirectly) as a result of the accreditation program. Overall, 37% of nonapplicant respondents strongly agreed or agreed that their health department had experienced other benefits, either directly or indirectly, as a result of accreditation. Sixty-two percent of nonapplicants that reportedly plan to apply for accreditation strongly agreed or agreed compared with only 7% of respondents that reportedly did not plan to apply and 25% of respondents that were reportedly undecided about applying for PHAB accreditation.
Comparison of perceived QI engagement between nonapplicants and accredited health departments
Both the nonapplicant and postaccreditation surveys included questions regarding QI knowledge and infrastructure. We analyzed responses to these questions to assess differences between the nonapplicants and accredited health departments. Nonapplicant respondents were more likely to indicate that their health department staff had either no knowledge or only familiarity with QI (48% reported no knowledge, subset of staff having familiarity, or the majority of staff having familiarity). In comparison, among health departments accredited for 1 year, 17% said their health department staff had no knowledge or only familiarity with QI. The accredited health departments were more likely to report that a majority of staff were knowledgeable and practiced QI or routinely practiced QI. Among the health departments accredited for 1 year, 49% said the majority of their staff were knowledgeable of QI or routinely practiced QI compared with 21% of nonapplicant health departments. See Table 3.
When asked about QI infrastructure, 44% of nonapplicant health departments said that QI was conducted informally or not at all. In the postaccreditation survey, only 7% said that QI was conducted informally or not at all.
Nonapplicants' reported barriers to pursuing accreditation
Nonapplicant survey respondents who reported that they were not applying for PHAB accreditation were asked to indicate the reasons for deciding not to pursue accreditation. The most common reasons for not applying were as follows: time and effort required for accreditation exceeded benefits (88%), and fees for accreditation were too high (81%). Nonapplicants were also asked to describe the extent to which they experienced challenges related to accreditation. Among all nonapplicant survey respondents, 37% agreed or strongly agreed that their health department had experienced challenges (directly or indirectly) as a result of the accreditation program. Among nonapplicant health departments that planned to apply for accreditation, 57% agreed or strongly agreed that they had experienced challenges resulting from accreditation compared with 11% that had decided not to apply and 29% that were undecided.
Perceived changes in QI knowledge and practice were described by nonapplicant survey respondents following an open-ended survey question. In total, 26 respondents provided a written response to the open-ended question, describing the types of changes and how the PHAB accreditation program contributed to the change. Among the 26 responses, 8 mentioned that their health departments were implementing QI projects or developing their QI infrastructure. For example, one respondent said, “We are aware that QI and performance management initiatives are a significant component of accreditation readiness and are working steadily to improve our knowledge and skills in QI.” Seven responses mentioned a perceived increase in awareness of QI. Five responses mentioned QI training, and 3 said there had been an increased discussion of or interest in QI at the management or leadership level. Two responses specifically mentioned using PHAB requirements to inform QI processes. For example, one respondent said, “We have systematically reviewed all PHAB domains, standards, and measures and compared them against our current practices. Our leadership team used the PHAB material for suggestions on ways we could improve our operations.” Finally, 4 respondents reported minimal or no changes in terms of QI activities in their health departments.
Nonapplicant survey respondents were also asked to provide any additional information in terms of any benefits or challenges related to accreditation. Three of the 6 responses that mentioned benefits specifically described benefits relating to the CHA, CHIP, and strategic plan. One respondent said, “There has been an increased awareness of the health district and our work because of the CHA/CHIP process and the coalition building.” Another respondent stated, “Whether accredited or not, the organizational management and community assessment and planning aspects of the accreditation requirements improve the quality of the [health department].”
Of the 15 nonapplicants that provided a written response to the open-ended survey question related to challenges, 4 respondents described not having enough staff to undertake accreditation activities and 3 said they lacked the funding necessary for accreditation. Five respondents stated that the resources (eg, staff, time, funding) required for accreditation would negatively impact or had already negatively impacted service provision. Resource reallocation for accreditation can have a ripple effect; for example, one respondent stated, “These additional expenses and a decrease in services are starting to have a negative impact on our credibility throughout the community.” Additional challenges reported by nonapplicants included accreditation leading to staff burnout and stress, a lack of staff desire to participate in accreditation efforts due to the amount of extra work it would require, and difficulty aligning the health department's strategic plan with PHAB accreditation standards.
Findings from the nonapplicant survey indicate that the accreditation program has affected the broader public health field in several ways. First, as public health departments conduct CHAs, CHIPs, and strategic plans, they reference PHAB requirements for these assessment and strategic planning activities. The majority of nonapplicant health departments that planned to apply for accreditation, half of those that were undecided about applying and nearly one-third of those that were not applying for accreditation were reportedly referencing PHAB requirements when developing these documents. Second, as nonapplicants assess their readiness to apply for accreditation, certain benefits may be attributed to accreditation. The majority of nonapplicants that intended to apply for accreditation but had not yet submitted their application reported that PHAB accreditation had helped their health department identify their strengths and weaknesses and had increased their health department's focus on QI efforts. Around half of the health departments that were undecided about applying also believed that PHAB accreditation had affected these benefits in their health department, showing that the benefits extend beyond health departments that were accredited. By assessing their readiness for accreditation and considering whether or not to apply, which involved evaluating current health department processes and structures, health departments could achieve certain perceived benefits prior to officially beginning the accreditation processes. These perceived benefits were not reported among health departments that did not plan to apply for accreditation.
The findings also demonstrate that 1 year following accreditation, accredited health departments are more likely to report a higher level of staff knowledge of QI and a stronger QI infrastructure within their agency, than nonapplicants. Positive changes have been observed within health departments that implement QI activities, including improvements in efficiency and effectiveness.11–15 While these results indicate that accredited health departments are more likely than nonapplicants to engage in formal QI activities, those that pursue accreditation are the most likely to have the capacity to undertake these QI initiatives. Capacity is a major barrier and the time, effort, and fees required to apply for accreditation continue to be the most common factors health departments report as their reasons for not pursuing accreditation.2–4
There are several limitations to the findings presented. First, the nonapplicant survey represents a small subset of the total number of health departments in the United States. The low response rate for the nonapplicant survey limits the generalizability of the findings, and results may not be representative of the broader universe of health departments. In addition, we did not conduct analyses to indicate whether differences in the samples were statistically significant. Second, nonapplicant health departments that were currently considering undergoing accreditation may have been more likely to respond to the survey and may be more predisposed to have positive perceptions of PHAB and accreditation than other nonapplicants. This also likely explains the high percentage of survey respondents that indicated they plan to apply for accreditation in comparison with the NACCHO Profile survey findings. Third, the survey questions regarding perceived benefits of accreditation are subjective and therefore may be influenced by the perspective of the individual who completed the survey. Finally, a limited number of respondents provided qualitative data in the open-ended responses, so these data are intended to be illustrative examples and are not generalizable to the larger population of LHDs.
Implications for Policy & Practice
- Nonapplicant health departments are aware of PHAB and reference its accreditation guidelines, regardless of the intent to apply. PHAB has the ability to reach and influence a wide range of health departments with their materials.
- Health departments may experience benefits associated with accreditation prior to formal involvement in PHAB accreditation. These perceived benefits include increased awareness of strengths and weaknesses, stimulated QI and performance improvement opportunities, and increased awareness of/focus on QI efforts.
- The most common challenges health departments face when applying for accreditation are identifying the time and resources to dedicate to the process.
3. Shah GH, Leep CJ, Sellers K, Liss-Levinson R, Williams KS. Public health
agencies' level of engagement in and perceived barriers to PHAB national voluntary accreditation
. J Public Health
Manag Pract. 2015;21(2):107–115.
4. Yeager VA, Ye J, Kronstadt J, Robin N, Leep CJ, Beitsch L. National voluntary public health accreditation
: are more local health departments intending to take part? J Public Health
Manag Pract. 2016:22(2):149–156.
5. Kronstadt J, Meit M, Siegfried A, Nicolaus T, Bender K, Corso L. Evaluating the impact of national public health
—United States, 2016. MMWR Morb Mortal Wkly Rep. 2016;65(31):803–806.
6. Siegfried A, Heffernan M, Kennedy J, Meit M. Quality improvement
and performance management benefits of public health accreditation
: national evaluation
findings. Supplement, Impact of Public Health Accreditation
. J Public Health
Manag Pract. 2018;24(suppl 3):S3–S9.
11. Riley WJ, Moran JW, Corso LC, Beitsch LM, Bialek R, Cofsky A. Defining quality improvement
in public health
. J Public Health
Manag Pract. 2010;16(1):5–7.
12. Dilly JA, Bekemeier B, Harris JR. Quality improvement
interventions in public health
systems: a systematic review. Am J PrevMed. 2012;42(5)(suppl 1):S58–S71.
13. Riley W, Parsons H, McCoy K, et al Introducing quality improvement
methods into local public health
departments: structured evaluation
of a statewide pilot project. Health Serv Res. 2009;44(5, pt 2):1863–1879.
14. Harrison LM, Shook ED, Harris G, Lea CS, Cornett A, Randolph GD. Applying the model for improvement in a local health department: quality improvement
as an effective approach in navigating the changing landscape of public health
practice in Buncombe County, North Carolina. J Public Health
Manag Pract. 2012;18(1):19–26.
15. McLees AW, Nawaz S, Thomas C, Young A. Defining and assessing quality improvement
outcomes: a framework for public health
. Am J Public Health
. 2015;105(suppl 2):S167–S173.