For more than 10 years, the Public Health Accreditation Board (PHAB) has been working with national public health leaders and agencies to develop and execute a national voluntary accreditation process. These efforts arose in response to a call to improve the quality and performance of the public health system and its workforce.1–3 Accreditation is a multiyear process that requires health departments to demonstrate participation in core public health activities as well as engagement with their workforce in assessing training needs and providing training in response.2 As of June 2018, 31 state health agencies (SHAs), 191 local health departments (LHDs), and 1 tribal health department have been accredited, along with 1 integrated local public health system comprising 67 LHDs in 1 centralized state. Another 9 SHAs and 184 LHDs are formally involved in the process.4 While much progress has been made (69% of the US population is covered by an accredited SHA and/or LHD), questions about the value and impact of accreditation are the current focus of attention and may be factors for agencies not yet involved in accreditation.3
Numerous early studies examined progress toward national accreditation and informed strategies to support HDs in becoming accredited.5–9 The majority of these studies focused on LHD characteristics related to participation or intent to participate in accreditation and generally found that smaller LHDs or those in rural areas were less likely to participate.7–9 More recent studies have attempted to ascertain the value of national voluntary accreditation.10–18 For the most part, these studies relied on self-reported perspectives of individuals closely involved in the accreditation process.12–14 They typically reported that it adds value and provides a benefit to their HDs. However, the majority of these studies have been limited to LHDs13–17 and/or are limited by social desirability bias.12–14,18 New contributions, such as the recent study by Ye and colleagues,15 address some concerns about social desirability bias by assessing what accreditation means to the workforce as a whole, rather than just to those closely involved. Additional studies that examine accreditation at the SHA level are needed along with studies that assess the impact on the workforce in a more generalizable way. Such studies are especially warranted, given long-held concerns by some that the accreditation process is onerous and may further burden the already understaffed and overwhelmed public health workforce.
This cross-sectional study utilizes 2017 Public Health Workforce Interests and Needs Survey (PH WINS) data from governmental public health workers to examine the relationship between accreditation status and employee satisfaction, intention to leave, awareness of public health concepts, and training needs among both state and local HD staff. The study controls for individual and organizational factors that have been found to influence accreditation or workforce satisfaction. This study presents the most generalizable perspectives of the public health workforce regarding the relationship between accreditation and workforce outcomes to date. Findings will be of interest to national leaders, SHAs/LHDs, and others involved in accreditation and policy-making.
Methods
Study design, data, and population studied
This cross-sectional study utilizes secondary data of training needs, satisfaction, and awareness of public health concepts of the governmental public health workforce from PH WINS 2017. PH WINS is a national survey designed and administered by the Association of State and Territorial Health Officials (ASTHO) with support from the de Beaumont Foundation. The data include a nationally representative sample of state and local public health employees; demographic and employment information; and respondents' perceptions about their experience, training needs, satisfaction, intentions to leave, and awareness of public health concepts. A total of 47 604 respondents were considered for inclusion from PH WINS 2017, representing a 48% response rate.
The PH WINS data were linked to a Public Health Accreditation Board (PHAB) dataset of agency accreditation status. Accreditation status was categorized as not engaged in accreditation, formally involved in accreditation (defined for this study as having submitted their registration in the accreditation system), or accredited as of August 2017 (1 month prior to the initial fielding of PH WINS). The LHDs that were part of an application comprising multiple agencies (including a singular state application on behalf of a centralized state's local agencies or a multijurisdictional application of 2 or more local agencies, n = 6184) were excluded from the local sample as their experiences may differ from those of agencies that applied/received accreditation individually. In addition, respondents who identified their role as clerical personnel, custodian, or who were part-time employees (n = 3324), or other (n = 10 890) were excluded from analyses as their roles are less influenced by accreditation or by the training areas assessed in PH WINS. Individuals missing key demographic variables of interest were also removed (4018). Some individuals met more than one of these exclusion criteria. In total, 19 614 individuals were removed from the sample.
Dependent variables
We considered outcomes that are theoretically associated with accreditation based on the accreditation logic model and previous research.2 Specifically, we considered the relationship between accreditation and 3 general constructs: training needs, satisfaction/burnout/intent to leave, and awareness of public health concepts.
Training needs
Training variables included binary measures for the following individual perceptions: whether training needs are assessed, whether the individual has a gap in a specific skill, and the overall presence of any training gap. To determine whether an individual's training needs are assessed, respondents were asked to evaluate the statement “my training needs are assessed” on a 5-point Likert scale. The 2017 version of PH WINS listed 21 to 22 specific skills for each of the 3 supervisory tiers (tier 1: nonsupervisor, tier 2: mid-level [supervisor/manager], or tier 3: executive]. Individuals were asked first about how important each specific skill is to their daily work, followed by their perceived skill level. Importance and skill level were both assessed on a 4-point Likert scale with an additional “not applicable” option for skill level. A gap was indicated when importance was reported as “somewhat” or “very important” and skill level was reported as “unable to perform” or “beginner.” If at least 1 of the 21 to 22 specific skills revealed a gap, that individual was coded as having any training gap.
Generally, the specific skills assessed across the 3 supervisory tiers were similar but reflected specific expectations of that tier. For example, for skills related to business plans, nonsupervisors were asked about the ability and importance of “describing the value of an agency business plan” while mid-level employees were asked about “implementing a business plan for agency programs and services” and executives about “designing a business plan for the agency.”
Satisfaction/burnout
Satisfaction and burnout were measured through individual items as well as a burnout inventory. Satisfaction with job and satisfaction with organization were asked on a 5-point Likert scale. Responses for both satisfaction variables were dichotomized where “somewhat satisfied” and “very satisfied” were coded together as satisfied and “neither,” “somewhat dissatisfied,” and “very dissatisfied” were coded as not satisfied.19
Burnout was measured using a series of items that comprise the Oldenburg Burnout Inventory (OLBI) (ie, exhaustion, disengagement, overall burnout).20 Individual factors that contributed to each of the 3 items were considered on a 5-point Likert scale. The 3 summative items themselves (eg, exhaustion, disengagement, and overall burnout) were then converted to a 4-point scale with 2.5 for a response of “neither agree nor disagree.” To create a binary indicator for burnout, a complete OLBI score of 2.5 or less was considered “no burnout” while any score greater than 2.5 was considered “burnout.” We conducted sensitivity analyses to explore whether the way we grouped “burnout” versus “other” mattered, and we did not find any differences (see Supplemental Digital Content Appendix 2, available at https://links.lww.com/JPHMP/A547). Individuals were also asked about their intentions to leave within the next year and those who reported intending to leave were asked to select why from a list of reasons. Two intention-to-leave variables were examined specifically: intend to leave in the next year for any reason (aside from retirement) and intend to leave because of burnout or lack of job satisfaction.
Awareness of public health concepts
The PH WINS also asked about individual awareness of public health concepts including quality improvement (QI), evidence-based public health practice, health in all policies, public health and primary care integration, multisectoral collaborations, and cross-jurisdictional sharing of public health services. Individuals responded on a 4-point Likert scale. A new variable for awareness was generated for each concept by dichotomizing responses into those answering “a lot” whereas those who said “a little,” “not much,” or “nothing at all” were grouped together. Only “a lot” was considered awareness because of possible social desirability bias that may influence respondents to report at least “a little” awareness even when not present.
Independent variables
The primary independent variable was agency accreditation status, categorized as not engaged, formally involved in the accreditation process, or accredited. Individual respondent variables included gender, race/ethnicity (white, black, or African American; Hispanic or Latino; or other), age (10-year brackets), highest degree obtained (no degree or associates, bachelors, masters, or doctorate), having formal public health training as defined by a bachelor's degree or above in a public health field, and tenure within the current agency (5-year brackets). Other individual job characteristics included supervisory tier and role type (administrative, clinical or laboratory, public health sciences, or social services). In addition to accreditation status, agency characteristics included jurisdiction (state or local), state governance (decentralized, centralized, shared, or mixed), and agency size (tertiles).
Analysis
Descriptive statistics were performed for individual, employment, and agency characteristics by accreditation status using χ2 tests. Bivariate analyses of training needs, satisfaction/burnout, and awareness of public health trends were conducted using pairwise χ2 comparisons. Because of the multiple comparisons, alpha was adjusted according to the Bonferroni correction and set at .0167 for these analyses. Multivariate analyses of training needs, satisfaction/burnout, and awareness of public health concepts were all conducted using logistic regressions controlling for all individual, employment, and agency characteristics described previously. All regression analyses were conducted separately for respondents at the SHA and LHD levels. Because of the complex sampling and survey design, national survey and replication weights were included in all analyses. Analyses were conducted using Stata Version 15 (College Station, TX).
Results
Descriptive statistics
The study sample included 24 083 respondents (104 928 weighted responses). Of the respondents 67% were from LHDs and 33% were from SHAs (Table 1). Among local respondents, 48% were from agencies that were either formally involved or accredited at the time of the survey compared with 80.1% of state respondents. A total of 328 LHDs (241 not engaged, 32 formally involved, and 55 accredited) and 46 SHAs (12 not engaged, 12 formally involved, and 22 accredited) were represented in these data.
TABLE 1 -
Descriptive Statistics by Accreditation Status and Agency Type, Weighted
a
|
Local Health Departments (n = 70 302) |
State Health Agencies (n = 34 626) |
| Never Involved (n = 36 552) |
Formally Involved (n = 12 194) |
Accredited (n = 21 556) |
P
|
Never Involved (n = 6865) |
Formally Involved (n = 9821) |
Accredited (n = 17 940) |
P
|
| Gender |
| Female |
27 042 (74.0) |
9418 (77.2) |
16 728 (77.6) |
.63 |
4631 (67.5) |
6936 (70.6) |
12 135 (67.6) |
.10 |
| Male |
9380 (25.7) |
2746 (22.5) |
4754 (22.1) |
|
2202 (32.1) |
2851 (19.0) |
5718 (31.9) |
|
| Other |
129 (0.4) |
30 (0.2) |
75 (0.3) |
|
32 (0.5) |
34 (0.3) |
87 (0.5) |
|
| Race/ethnicity |
| White |
23 730 (64.9) |
7659 (62.8) |
10 699 (49.6) |
.003 |
4283 (62.4) |
6130 (62.4) |
12 481 (69.6) |
<.001 |
| Black or African American |
7266 (19.9) |
1742 (14.3) |
3081 (14.3) |
|
596 (8.7) |
2053 (20.9) |
1660 (9.3) |
|
| Hispanic or Latino |
2683 (7.3) |
1630 (13.4) |
4093 (19.0) |
|
731 (10.6) |
704 (7.2) |
1310 (7.3) |
|
| Other |
2874 (7.9) |
1163 (9.5) |
3684 (17.1) |
|
1255 (18.3) |
934 (9.5) |
2488 (13.9) |
|
| Age, y |
| ≤25 |
824 (2.3) |
392 (3.2) |
584 (2.7) |
.13 |
125 (1.8) |
140 (1.4) |
329 (1.8) |
.01 |
| 26-35 |
5381 (14.7) |
2240 (18.4) |
4479 (20.8) |
|
1282 (18.7) |
1916 (19.5) |
2922 (16.3) |
|
| 36-45 |
8090 (22.1) |
2964 (24.3) |
5003 (23.2) |
|
1493 (21.7) |
2483 (25.3) |
4113 (22.9) |
|
| 46-55 |
11 891 (32.5) |
3207 (26.3) |
6302 (29.2) |
|
1870 (27.2) |
2592 (26.4) |
5112 (28.5) |
|
| 56-65 |
8364 (22.9) |
3084 (25.3) |
4671 (21.7) |
|
1787 (26.0) |
2417 (24.6) |
4837 (27.0) |
|
| ≥66 |
2002 (5.5) |
308 (2.5) |
518 (2.4) |
|
308 (4.5) |
273 (2.8) |
627 (3.5) |
|
| Education level |
| No college or associate's degree |
8844 (24.2) |
3318 (27.2) |
4555 (21.1) |
.45 |
1174 (17.1) |
1455 (14.8) |
3338 (18.6) |
<.001 |
| Bachelor's |
16 526 (45.2) |
5281 (43.3) |
9388 (43.6) |
|
2506 (36.5) |
3113 (31.7) |
6959 (38.8) |
|
| Master's |
9940 (27.2) |
3104 (25.5) |
6718 (31.2) |
|
2504 (36.5) |
4364 (44.4) |
5911 (32.9) |
|
| Doctorate |
1242 (3.4) |
491 (4.0) |
895 (4.2) |
|
680 (9.9) |
889 (9.1) |
1732 (9.7) |
|
| Public health degree |
| No PH degree |
31 656 (86.6) |
9955 (81.6) |
17 422 (80.8) |
.04 |
5330 (77.6) |
7006 (71.3) |
14 090 (78.5) |
<.001 |
| PH degree |
4896 (13.4) |
2238 (18.4) |
4134 (19.2) |
|
1535 (22.4) |
2815 (28.7) |
3850 (21.5) |
|
| Tenure in agency, y |
| 0-5 |
13 482 (36.9) |
5431 (44.5) |
9364 (43.4) |
.37 |
3062 (44.6) |
4695 (47.8) |
7577 (42.2) |
<.001 |
| 6-10 |
6465 (17.7) |
1579 (12.9) |
3804 (17.6) |
|
1162 (16.9) |
1864 (19.0) |
3124 (17.4) |
|
| 11-15 |
4700 (12.9) |
1767 (14.5) |
2949 (13.7) |
|
852 (12.4) |
1462 (14.9) |
2551 (14.2) |
|
| 16-20 |
6638 (18.2) |
1597 (13.1) |
2574 (11.9) |
|
656 (9.6) |
811 (8.3) |
1984 (11.1) |
|
| ≥21 |
5267 (14.4) |
1820 (14.9) |
2866 (13.3) |
|
1133 (16.5) |
990 (10.1) |
2704 (15.1) |
|
| Supervisory tier |
| Nonsupervisor |
24 177 (66.1) |
8253 (67.7) |
15 075 (69.9) |
.34 |
4448 (64.8) |
6040 (61.5) |
11 662 (65.0) |
.002 |
| Mid-level |
11 414 (31.2) |
3555 (29.2) |
5795 (26.9) |
|
2116 (30.8) |
3333 (33.9) |
5824 (32.5) |
|
| Executive |
961 (2.6) |
386 (3.2) |
686 (3.2) |
|
300 (4.4) |
448 (4.6) |
454 (2.5) |
|
| Role type |
| Administrative |
4593 (12.6) |
1855 (15.2) |
3520 (16.3) |
.47 |
1667 (24.3) |
2725 (27.7) |
4815 (26.8) |
<.001 |
| Clinical and laboratory |
13 583 (37.2) |
4019 (33.0) |
7912 (36.7) |
|
1443 (21.0) |
1699 (17.3) |
3759 (21.0) |
|
| Public health sciences |
16 090 (44.0) |
5560 (45.6) |
8739 (40.5) |
|
3333 (48.6) |
5122 (52.2) |
8814 (49.1) |
|
| Social services |
2286 (6.3) |
760 (6.2) |
1386 (6.4) |
|
422 (6.1) |
275 (2.8) |
552 (3.1) |
|
| Governance |
| Decentralized |
21 168 (57.9) |
8806 (72.2) |
18120 (84.1) |
.04 |
3353 (48.8) |
4446 (45.3) |
12 563 (70.0) |
<.001 |
| Centralized |
8773 (24.0) |
302 (2.5) |
908 (4.2) |
|
1925 (28.0) |
1715 (17.5) |
2037 (11.4) |
|
| Shared |
3621 (9.9) |
1908 (15.6) |
1659 (7.7) |
|
27 (0.4) |
2961 (30.1) |
2588 (14.4) |
|
| Mixed |
2990 (8.2) |
1178 (9.7) |
869 (4.0) |
|
1560 (22.7) |
699 (7.1) |
752 (4.2) |
|
| Agency size |
| Small |
3739 (10.2) |
470 (3.9) |
2 (0.0) |
.001 |
973 (14.2) |
967 (9.8) |
2060 (11.5) |
<.001 |
| Medium |
10 626 (29.1) |
2794 (22.9) |
2248 (10.4) |
|
700 (10.2) |
6568 (66.9) |
6077 (33.9) |
|
| Large |
22 187 (60.7) |
8930 (73.2) |
19 306 (89.6) |
|
5192 (75.6) |
2286 (23.3) |
9802 (54.6) |
|
aDifferences in characteristics by accreditation status were assessed using χ2 tests. The local health department sample descriptives were conducted separately from the state health agency sample. All frequencies reported are weighted estimates to be nationally representative of the governmental public work force.
Among the local respondents, race, presence of a public health degree, state governance style, and agency size in tertiles differed significantly by accreditation status (Table 1). Respondents from accredited or involved agencies generally tended to include greater proportions of Hispanics, Latinos, or those of other race/ethnicity and individuals with master's degrees. State respondents generally differed significantly by accreditation status for all demographic, job, and agency characteristics except for gender.
Bivariate analyses
Training needs
In bivariate analyses, significant differences generally identified a higher proportion of respondents reporting training gaps among formally involved respondents. In the local setting, compared with respondents in formally involved LHDs, fewer respondents in accredited agencies reported gaps in assessing external drivers in the environment that may influence the work (40.6% vs 34.1%, P = .01) (Table 2). In addition, compared with respondents in formally involved LHDs, fewer respondents in not engaged agencies reported engaging community assets and resources to improve health (38.2% vs 25.5%, P = .016). Among state respondents, differences existed in 3 training areas primarily between respondents in accredited versus not engaged agencies (Table 2). These included any training gap (61.4% among not engaged vs 63.3% among accredited, P = .004); describing agency priorities, mission, and vision (21.0% among not engaged vs 26.4% among accredited, P = .001); and engaging community assets and resources to improve health (28.5% among not engaged vs 33.9% among accredited, P = .006, 26.8% among formally involved vs 33.9% among accredited, P < .001).
TABLE 2 -
Bivariate Differences in Training Needs by Accreditation Status and Agency Type
a
|
Local Health Departments |
State Health Agencies |
| Not Engaged |
Formally Involved |
Accredited |
Not Engaged |
Formally Involved |
Accredited |
| Training outcome |
| Training needs are assessed |
56.9% |
57.9% |
59.7% |
53.7% |
49.4% |
52.1% |
| Any training gap |
61.8% |
68.6% |
66.6% |
61.4%b |
63.9% |
63.3%b |
| Tier 1 gap: Describe value of business plan |
38.2% |
48.6% |
44.9% |
38.2% |
37.9% |
40.5% |
| Tier 1 Gap: Assess external drivers in the environment that may influence the work |
29.7% |
40.6%c |
34.1%c |
26.4% |
28.1% |
30.4% |
| Tier 1 Gap: Describe agency priorities, mission, and vision |
30.0% |
31.0% |
25.3% |
21.0%b |
21.7% |
26.4%b |
| Tier 1 Gap: Engage community assets and resources to improve health |
25.5%d |
38.2%d |
30.3% |
28.5%b |
26.8%c |
33.9%b,c |
| Tier 2 Gap: Identify funding mechanisms and procedures for sustainable funding models |
41.2% |
50.5% |
46.1% |
38.4% |
32.9% |
33.2% |
| Satisfaction outcome |
| Satisfied with job |
82.9% |
82.3% |
83.2% |
80.8% |
82.0% |
79.3% |
| Satisfied with organization |
72.0% |
69.8% |
72.2% |
68.9% |
70.2% |
67.9% |
| Experiencing burnout (OLBI) |
26.8% |
27.7% |
23.8% |
27.6% |
26.9% |
27.4% |
| Intend to leave |
20.2% |
21.3% |
23.8% |
31.2% |
29.3% |
29.0% |
| Intend to leave because of burnout or lack of job satisfaction |
10.3% |
13.3% |
13.5% |
18.8% |
16.9% |
17.8% |
| Awareness of public health concepts |
| Quality improvement |
30.3%b |
40.3% |
45.9%b |
37.1%b,d |
49.0%c,d |
41.4%b,c |
| Evidence-based public health practice |
32.4% |
36.3%c |
43.0%c |
40.8%b |
44.0%c |
36.3%b,c |
| Health in all policies |
12.1% |
16.2% |
17.3% |
13.6% |
14.7% |
14.0% |
| Public health and primary care integration |
20.5% |
22.7% |
28.6% |
25.3%b |
26.9%c |
21.1%b,c |
| Multi-sectoral collaboration |
16.2%b |
19.8% |
23.7%b |
21.2%b |
23.6%c |
18.8%b,c |
| Cross-jurisdictional sharing of public health services |
19.7% |
17.3% |
19.8% |
17.0% |
17.2% |
16.2% |
Abbreviation: OLBI, Oldenburg Burnout Inventory.
aValues in table represent percentage of respondents within that agency type and accreditation status that exhibit the outcome. Significance was tested across accreditation status within a particular jurisdiction using pairwise χ2 tests for local and state health agency respondents separately. Because of the multiple comparisons, a Bonferroni adjustment was made and significance was set at .0167. Bold values indicate significant differences.
bSignificant difference between those not involved and those who are accredited.
cSignificant difference between those formally involved and those in accredited agencies. For example, within state health agency, a significantly greater proportion of respondents from accredited agencies reported at least 1 training gap (63.3%) compared with 61.4% of respondents from nonaccredited agencies. These proportions differed significantly although neither was significantly different from respondents from agencies formally involved in accreditation (63.9%).
dSignificant difference between not engaged and those formally involved in the accreditation process.
Satisfaction
No differences in satisfaction, intentions to leave, or burnout existed between respondents across accreditation status in either the state or local setting.
Awareness of public health concepts
Three differences were identified among local respondents in regard to awareness of public health concepts, with more respondents in accredited agencies reporting awareness of certain public health concepts. These included awareness of QI (30.3% among those not engaged vs 45.9% among accredited, P = .007); awareness of evidence-based public health practice (36.3% among those formally involved vs 43.0% among accredited, P = .016); and awareness of multisectoral collaboration (16.2% among not engaged vs 23.7% among accredited, P = .006).
Numerous differences existed between respondents in SHA settings in regard to awareness of public health concepts. For all concepts except for health in all policies and cross-jurisdictional sharing of public health services, respondents in formally involved agencies had a higher proportion of respondents who reported awareness than those in accredited and not engaged agencies (Table 2).
Multivariate analyses
Training needs
Individuals from LHDs did not report differences in having their training needs assessed or having any training gaps by accreditation status after controlling for demographics, job, and agency characteristics (Table 3). Individuals from formally involved SHAs had lower odds of reporting having their training needs assessed (P = .003) and those from accredited state agencies had greater odds of reporting any training gap (P = .03) relative to those from nonengaged agencies.
TABLE 3 -
Differences in Training Needs by Accreditation Status
a
| Training Outcome |
Accreditation Status |
LHD |
SHA |
| OR |
P
|
OR |
P
|
| Training needs are assessed |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
0.98 |
.94 |
0.82 |
.003 |
|
Accredited |
1.17 |
.27 |
0.94 |
.294 |
| Any training gap |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.35 |
.23 |
1.08 |
.25 |
|
Accredited |
1.29 |
.26 |
1.08 |
.03 |
| Tier 1 Gap: Describe value of business plan |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.64 |
.01 |
1.05 |
.60 |
| Accredited |
1.39 |
.01 |
1.13 |
.17 |
| Tier 1 Gap: Assess external drivers in the environment that may influence the work |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.57 |
.03 |
1.17 |
.19 |
| Accredited |
1.18 |
.44 |
1.26 |
.04 |
| Tier 1 Gap: Describe agency priorities, mission, and vision |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.11 |
.54 |
1.29 |
.009 |
| Accredited |
0.84 |
.41 |
1.50 |
<.001 |
| Tier 1 Gap: Engage community assets and resources to improve healthb |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.89 |
.008 |
1.00 |
.99 |
| Accredited |
1.36 |
.26 |
1.33 |
.003 |
| Tier 2 Gap: Identify funding mechanisms and procedures for sustainable funding modelsb |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.44 |
.24 |
0.75 |
.05 |
| Accredited |
0.22 |
.48 |
0.80 |
.03 |
Abbreviations: LHD, local health department; OR, odds ratio; SHD, SHA, state health agency.
aEstimates provided are the odds ratios for the given outcome based on either being formally involved in the accreditation process or accredited at the time of the survey compared with organizations that were neither involved nor accredited. Each outcome represents the results of a different logistic regression model where the state health agency and local health department models were run independently. Odds ratios greater than 1 indicate that there are greater odds of a given outcome compared with the reference group (not accredited respondents). Odds ratios less than 1 indicate lower odds of that outcome. All models included individual, job, and agency covariates as controls in the model. Individual covariates included age, gender, race/ethnicity, education level, formal public health degree, and tenure in agency. Job characteristics included supervisory tier and role type. Agency characteristics included state governance type and agency size in tertiles.
bBecause of sampling weights, covariates in this model were not consistently estimated (Stata BRR error).
Among tier 1 nonsupervisory local respondents, those who were formally involved were at greater odds of identifying a gap in describing the value of the agency business plan (P = .01), assessing the external drivers in their environment that may influence their work (P = .01), and engaging community assets and resources to improve health in a community (P = .008) than those from nonengaged agencies. Those from accredited agencies also had greater odds of reporting the gap in assessing the value of the agency business plan (P = .01).
Among tier 1 nonsupervisory state respondents, those from accredited agencies had greater odds of reporting gaps in assessing external drivers in their environment that may influence their work (P = .04), describing priorities, mission, and vision of the agency (P < .001), and engaging community assets to improve health (P = .003). Those from agencies formally involved in accreditation were also at greater odds of reporting a gap in describing agency priorities, mission, and vision (P < .001). Among tier 2 mid-level state respondents, those from accredited agencies had lower odds of reporting a gap in identify funding mechanisms and procedures for sustainable funding models (P = .03).
Because skill gaps were defined by individuals reporting both that they had low skill in a particular area and that it was of high importance to their day-to-day work, we considered whether the increased gaps were due to being more likely to report either low skill or high importance. Except for state respondents being at a higher odds of reporting low skill in describing the agency priorities, mission, and vision, all other skills showed no difference in low skill and either no difference or less odds of reporting high importance (see Supplemental Digital Content Appendix 1, available at https://links.lww.com/JPHMP/A547).
Satisfaction and burnout
At the local level, there were no significant differences in satisfaction, burnout, or intention to leave across accreditation status after controlling for individual and agency characteristics (Table 4). Among state respondents, being involved in the accreditation process or being accredited was associated with lower odds of being satisfied with their job (P = .01 and P = .002, respectively) and satisfied with the organization (P < .004). There were no differences in experiencing burnout or intentions to leave by accreditation status for state respondents.
TABLE 4 -
Differences in Satisfaction, Burnout, and Intentions to Leave by Accreditation Status
a
| Satisfaction Outcome |
Accreditation Status |
LHD |
SHA |
| OR |
P
|
OR |
P
|
| Satisfied with job |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
0.94 |
.74 |
0.84 |
.01 |
|
Accredited |
1.02 |
.90 |
0.84 |
.002 |
| Satisfied with organization |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
0.82 |
.26 |
0.82 |
.004 |
|
Accredited |
0.95 |
.73 |
0.85 |
.004 |
| Experiencing burnout (OLBI) |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
0.97 |
.89 |
1.05 |
.60 |
|
Accredited |
0.80 |
.18 |
0.96 |
.64 |
| Intend to leave |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.12 |
.71 |
0.94 |
.29 |
|
Accredited |
1.25 |
.26 |
0.93 |
.13 |
| Intend to leave because of burnout or lack of job satisfaction |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.39 |
.32 |
0.99 |
.93 |
| Accredited |
1.30 |
.24 |
0.99 |
.92 |
Abbreviations: LHD, local health department; OLBI, Oldenburg Burnout Inventory; OR, odds ratio; SHA, state health agency.
aEstimates provided are the odds ratios for the given outcome based on either being formally involved in the accreditation process or accredited at the time of the survey compared with organizations that were neither involved nor accredited. Each outcome represents the results of a different logistic regression model where the state health agency and local health department models were run independently. Odds ratios greater than 1 indicate that there are greater odds of a given outcome compared with the reference group (not accredited respondents). Odds ratios less than 1 indicate lower odds of that outcome. All models included individual, job, and agency covariates as controls in the model. Individual covariates included age, gender, race/ethnicity, education level, formal public health degree, and tenure in agency. Job characteristics included supervisory tier and role type. Agency characteristics included state governance type and agency size in tertiles.
Awareness of public health concepts
In adjusted models, local respondents who were accredited and state respondents who were formally involved or accredited were at higher odds of reporting a lot of awareness of QI than those from nonengaged agencies (P = .01, P = .001, and P = .006 respectively) (Table 5). Individuals from accredited SHAs were at lower odds of being aware of evidence-based public health practices (P = .02) but greater odds of being aware of health in all policies (P = .03). Neither of these differed significantly at the local level. Awareness of multisectoral collaboration, public health and primary care integration, and cross-jurisdictional sharing of public health services did not differ significantly by accreditation status for either state or local respondents.
TABLE 5 -
Differences in High Awareness of Public Health Concepts by Accreditation Status
a
| Public Health Concept |
Accreditation Status |
LHD |
SHA |
| OR |
P
|
OR |
P
|
| Quality improvement |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.54 |
.17 |
1.25 |
.001 |
|
Accredited |
1.99 |
.01 |
1.14 |
.006 |
| Evidence-based public health practice |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.12 |
.65 |
0.97 |
.69 |
|
Accredited |
1.44 |
.11 |
0.85 |
.02 |
| Health in all policies |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.34 |
.37 |
0.97 |
.65 |
|
Accredited |
1.32 |
.23 |
1.14 |
.03 |
| Multisectorial collaboration |
Not engaged |
Reference |
|
Reference |
|
|
Formally involved |
1.15 |
.55 |
1.02 |
.81 |
|
Accredited |
1.34 |
.13 |
0.92 |
.16 |
| Public health and primary care integration |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
1.09 |
.75 |
1.00 |
.95 |
| Accredited |
1.41 |
.18 |
0.87 |
.08 |
| Cross-jurisdictional sharing of public health services |
Not engaged |
Reference |
|
Reference |
|
| Formally involved |
0.87 |
.31 |
0.83 |
.06 |
| Accredited |
0.97 |
.77 |
0.91 |
.36 |
Abbreviations: LHD, local health department; OR, odds ratio; SHA, state health agency.
aEstimates provided are the odds ratios for the given outcome based on either being formally involved in the accreditation process or accredited at the time of the survey compared with organizations that were neither involved nor accredited. Each outcome represents the results of a different logistic regression model where the state health agency and local health department models were run independently. Odds ratios greater than 1 indicate that there are greater odds of a given outcome compared with the reference group (not accredited respondents). Odds ratios less than 1 indicate lower odds of that outcome. All models included individual, job, and agency covariates as controls in the model. Individual covariates included age, gender, race/ethnicity, education level, formal public health degree, and tenure in agency. Job characteristics included supervisory tier and role type. Agency characteristics included state governance type and agency size in tertiles.
Discussion
This study explores differences in health department workforce by accreditation status and is the first to do so using a large sample of both SHA and LHD employees. Descriptive statistics suggest that there are many significant differences in those who comprise the workforce across the 3 levels of engagement in accreditation. Despite these differences, bivariate analyses identified just a few differences in terms of training gaps and none related to satisfaction. Differences were more commonly identified concerning awareness of various public health concepts, especially among respondents in state agencies.
Multivariate analyses identified a handful of measures correlated with accreditation status. For many outcome variables of interest—including intent to leave and burnout—there were no significant differences. This may dispel some concerns about potential unintended consequences of accreditation. Some of these associations identified, however, were not anticipated. These may warrant further investigation and may point to opportunities to strengthen aspects of the accreditation program. For example, employees in formally involved SHAs were less likely to indicate that their training needs were assessed than those in uninvolved SHAs. This is unexpected, given the PHAB requirement for workforce development planning, including an assessment of staff needs. Yet, because these agencies were still undertaking the process, it is possible that needs assessments were planned but not yet conducted.
Similarly, employees of formally involved and accredited SHAs were more likely to identify gaps in their ability to describe agency priorities, mission, and vision, and accredited state employees were more likely to identify gaps in engaging community assets and resources to improve health. Given the emphasis in the accreditation requirements related to strategic planning and community engagement, this is worth further analysis. One hypothesis that should be explored is whether, as employees become more involved in community health improvement work, they become more aware of its complexities and therefore are more aware of how much additional opportunity exists to build their skills. This has been seen in QI: as greater understanding was acquired, greater deficits in training were acknowledged.5,21 A similar hypothesis could also explain some differences in having “a lot” of awareness about some public health concepts. In other words, could this be a case of the following: the more you know, the more you realize how much you still don't know. It is also possible that given the amount of subspecialization that exists, particularly in larger organizations, some of the accreditation-preparation work does not permeate throughout the entire health department. This is a distinct possibility particularly among large SHAs and may merit future investigation. Might accreditation strategies be designed to go deeper into the organization as a possible counter to the subspecialization trend?
Prior research has found that LHD employees in agencies formally engaged with accreditation had higher levels of job satisfaction.15 The current study found no difference by accreditation status in satisfaction among local employees. However, accredited and formally involved state employees showed lower levels of job satisfaction than those not engaged in accreditation. Further analysis is needed to better understand the nature of this relationship. Interestingly, state employees did not differ by accreditation status in terms of burnout. This appears to discount the theory that accreditation reduces job satisfaction by overburdening employees.
The most consistent correlations in the awareness of public health concepts outcomes are greater awareness of QI among staff of accredited and formally involved SHD and LHD agencies. This is consistent with prior research regarding accreditation's prominent role in increasing engagement in QI and a QI culture.16,18
This study highlights several opportunities for bolstering the way accreditation can strengthen the public health workforce. When PHAB released Version 1.5 of the Standards & measure in 2013, a new Measure was included—Measure 8.2.4 focused on a “work environment that is supportive to the workforce.”22 While nearly 80% of the health departments that were accredited as of August 2017 used the prior version (1.0) of the Standards that did not include this measure, agencies pursuing accreditation now using Version 1.5 and those seeking reaccreditation will be required to implement practices designed to improve the work setting. This holds the potential for promoting stronger job and organizational satisfaction in the future. In addition, PHAB is currently developing Version 2.0 of the Standards & Measures.23 Findings from evolving research can inform those revisions maximizing accreditation's role in promoting a stronger workforce.
Further analyses using longitudinal data and more rigorous study designs should be implemented to better assess the causal effects of accreditation on the workforce. Studies exploring differences in structure, services, and various measures of agency effectiveness and community health should be considered to assess other potential effects of accreditation that reflect the goals of accreditation and the logic model.
Strengths and limitations
While the current study has several limitations, it also improves upon prior work in this realm. By utilizing a large, national sample and including individuals across agencies instead of only those most closely involved in accreditation, our results are more representative of the public health workforce. However, an important limitation is that these analyses are based on cross-sectional data and therefore cannot assess any causal effects of accreditation. It cannot be determined whether these findings are the result of baseline differences in training gaps, satisfaction/burnout/intentions to leave, and awareness of trends in an agency. Similarly, our analysis cannot account for the possible selection bias of health departments choosing to participate in the accreditation process. Health departments may decide to undergo accreditation for a variety of reasons, one of which may be to improve upon existing gaps exhibited by the health department exhibits. A second limitation is that due to the early stage of accreditation and research surrounding workforce changes, we tested many comparisons and multivariate models. With the significance level set at .05, we would expect to find 5% of tests to be significant by chance and therefore we risk type 1 error in reporting significant differences that may not in fact exist. Third, as Table 1 demonstrates, the workforce composition and HD characteristics vary significantly across the 3 accreditation categories—particularly among SHAs. Although this analysis statistically controlled for many observable differences, it is possible that there are other factors that were not included in this model that may account for differences, rather than accreditation status. Finally, while we considered examining LHD employees in Big City Health Coalitions and non–Big City Health Coalitions separately, it was not possible due to the PH WINS complex survey design and the size of the sample of respondents from non–Big City Health Coalitions LHDs.
Implications for Policy & Practice
- Differences were identified in the composition of the health department workforce across accredited, formally involved, and nonengaged health departments, as well as in several measures related to training, satisfaction, and awareness. A longitudinal analysis is needed to further tease out these differences and to better understand how undergoing the accreditation process may—or may not—contribute to them.
- There are opportunities to improve job and organizational satisfaction, particularly among SHA employees in the cohort of agencies accredited as of August 2017. Deliberate emphasis on strengthening the work environment, as is required in Version 1.5 of the accreditation Standards & Measures, may be needed.
- Care should be taken to ensure that efforts to demonstrate conformity with national standards permeate throughout the staff of the agency, rather than be focused on the work of a small group. This should be considered as part of efforts to revise the Standards & Measures and continue to improve the value of the accreditation process.
References
1. Riley WJ, Bender K, Lownik E. Public health department accreditation implementation: transforming public health department performance. Am J Public Health. 2012;102(2):237–242.
2. Public Health Accreditation Board. Public health agency system logic model. Available at
http://www.phaboard.org/wp-content/uploads/Accreditation-LogicModel-201706.pdf. Published June 2017. Accessed June 11, 2018.
3. Kronstadt J, Beitsch LM, Bender K. Marshaling the evidence: the prioritized public health accreditation research agenda. Am J Public Health. 2015;105(S2):S153–S158.
4. Public Health Accreditation Board. Accreditation activity as of May 21, 2018.
http://www.phaboard.org/news-room/accreditation-activity/. Published 2018. Accessed June 27, 2018.
5. Madamala K, Sellers K, Beitsch LM, Pearson J, Jarris P. Quality improvement and accreditation readiness in state public health agencies. J Public Health Manag Pract. 2012;18(1):9–18.
6. McLees AW, Thomas CW, Nawaz S, Young AC, Rider N, Davis M. Advances in public health accreditation readiness and quality improvement: evaluation findings from the national public health improvement initiative. J Public Health Manag Pract. 2014;20(1):29–35.
7. Yeager VA, Ye J, Kronstadt J, Robin N, Leep CJ, Beitsch LM. National voluntary public health accreditation: are more local health departments intending to take part? J Public Health Manag Pract. 2016;22(2):149–156.
8. Shah GH, Beatty K, Leep C. Do PHAB accreditation prerequisites predict local health departments' intentions to seek voluntary national accreditation. Front Public Health Serv Syst Res. 2013;2(3):Article 4.
9. Yeager VA, Ferdinand AO, Beitsch LM, Menachemi N. Local public health department characteristics associated with likelihood to participate in national Accreditation. Am J Public Health. 2015;105(8):1653–1659.
10. Yeager VA, Wharton K, Beitsch LM. Maintaining a competent public health workforce: lessons learned from experiences with public health accreditation domain 8 standards and measures [published online ahead of print January 10, 2018]. J Public Health Manag Pract. doi: 10.1097/PHH.0000000000000750.
11. Bender KW, Kronstadt JL, Wilcox R, Tilson HH. Public health accreditation addresses issues facing the public health workforce. Am J Prev Med. 2014;47(5):S346–S351.
12. Kittle A, Liss-Levinson R. State health agencies' perceptions of the benefits of accreditation. J Public Health Manag Pract. 2018;24(3):S98–S101.
14. Kronstadt J, Meit M, Siegfried A, Nicolaus T, Bender K, Corso L. Evaluating the impact of national public health department accreditation—United States, 2016. MMWR Morb Mortal Wkly Rep. 2016;65:803–806.
15. Ye J, Verma P, Leep C, Kronstadt J. Public health employees' perception of workplace environment and job satisfaction: the role of local health departments' engagement in accreditation. J Public Health Manag Pract. 2018;24(3):S72–S79.
16. Beitsch LM, Kronstadt J, Robin N, Leep C. Has voluntary public health accreditation impacted health department perceptions and activities in quality improvement and performance management? Supplement, Impact of Public Health Accreditation. J Public Health Manag Pract. 2018;24(suppl 3):S10–S18.
17. Ingram RC, Mayes GP, Kussainov N. Changes in local public health system performance before and after attainment of national accreditation standards. Supplement, Impact of Public Health Accreditation. J Public Health Manag Pract. 2018;24(suppl 3):S25–S34.
18. Siegfried A, Heffernan M, Kennedy M, Meit M. Quality improvement and performance management benefits of public health accreditation: national evaluation findings. Supplement, Impact of Public Health Accreditation. J Public Health Manag Pract. 2018;24(suppl 3):S3–S9.
19. Mazurenko O, Menachemi N. Environmental market factors associated with physician career satisfaction. J Healthc Manag. 2012;57(5):307–322.
20. Demerouti E, Bakker AB. The Oldenburg Burnout Inventory: a good alternative to measure burnout and engagement. Handbook of Stress and Burnout in Health Care. Hauppauge, NY: Nova Science; 2008.
21. Luo H, Sotnikov S, McLees A, Stokes S. Factors driving the adoption of quality improvement initiatives in local health departments: results from the 2010 profile study. J Public Health Manag Pract. 2015;21(2):176–185.
22. Public Health Accreditation Board. Standards & Measures Version 1.5. Alexandria, VA: Public Health Accreditation Board; 2013.
23. Bender K, Kronstadt J, Nicola B. Looking ahead: applying foresight principles to public health accreditation. J Public Health Manag Pract. 2018;24(suppl 3):S126–S128.
