Improving population health outcomes relies upon the capacity of the public health workforce to effectively deliver public health services.1–3 The challenges associated with accurately enumerating and characterizing the public health workforce, 1 of 4 key domains of the national research agenda for Public Health Services and Systems,4 are well documented.5–10 The lack of a systematic method for monitoring workforce size and composition is a major barrier to collecting data to generate reliable projections of current and future workforce supply and demand.5 Although several organizations have periodically surveyed certain professions and organizational units of the public health workforce,11–16 the field as a whole has lacked standardized terminology to describe worker characteristics. Recent attempts to enumerate the governmental public health workforce by occupation have utilized secondary data from varying sources; the estimations are limited by the inconsistencies in methodology across surveys, yielding results fragmented by profession and difficult to merge into a cohesive picture of the public health workforce.6 , 17 The development of a common taxonomy to consistently describe public health workers over time, across surveys, and for different organizational types has emerged as a need resulting from attempts to combine workforce data from multiple sources in producing a single national estimate.
In 2014, the University of Michigan Center of Excellence in Public Health Workforce Studies (UM CEPHS) published a public health workforce taxonomy developed by a national advisory group. This taxonomy provides a schema for categorizing workers to better standardize workforce data collection.18 It includes 12 distinct axes by which different worker characteristics are categorized: occupation, setting, employer, education, licensure, certification, job tasks, program area, public health specialization area/expertise, funding source, condition of employment, and demographics. During development of the taxonomy, the Association of State and Territorial Health Officials (ASTHO), which served as a member of the advisory group, concurrently began crafting a comprehensive survey targeting individual public health workers. The Public Health Workforce Interests and Needs Survey (PH WINS) is the first nationally representative survey of individual public health practitioners employed by state health agencies, and it also included a pilot survey of a limited number of local health department (LHD) workers.19
As the first national public health workforce survey to use the public health workforce taxonomy, PH WINS afforded an opportunity to incorporate the taxonomy into a study tool while testing its actual functionality in characterizing workers. Seven of the 12 taxonomy axes were used in the PH WINS survey instrument, 3 of which—occupation, certification, and program area—provided analyzable qualitative data when respondents chose “other.” The data obtained from “other” responses for these 3 axes are the focus of this analysis. The purpose of this study was to determine the relative utility, reliability, and accuracy of 3 axes of the public health workforce taxonomy in categorizing local and state public health workers using a survey tool. This specifically included the goal of reducing the number of responses classified as “other” occupation, certification, or program areas by recoding responses into taxonomy categories and determining potential missing categories for recommendation to the advisory committee that developed the taxonomy.
PH WINS was created to assess perceptions and needs of the public health workforce. The online survey included 3 distinct sample frames: (1) a nationally representative sample of state health agencies; (2) employees of members of the Big Cities Health Coalition, to which the country's largest LHDs belong; and (3) a pilot frame of LHDs. The LHD sample for the PH WINS study was not intended to be nationally representative but rather a pilot attempt at surveying individual LHD workers. Survey development and study design are further detailed in an accompanying article in this supplement issue.19
PH WINS used the following questions to obtain respondent data on occupation, certification, and program area, respectively: (1) Please identify the classification that best represents your role in the organization; (2) Please indicate which credentials you have obtained. Check all that apply; and (3) Please specify your primary program area. ASTHO used a draft version of the taxonomy to create response choices for these questions. The taxonomy's occupational, certification, and program area categories were used as response options for the PH WINS questions about role, credentials, and primary program area, respectively. Respondents were asked to provide additional information if they chose “other” for any of these questions. Data used in this analysis included all respondents who completed questions of interest related to the occupation, certification, and program area, regardless of whether they completed the whole survey. As such, our final respondent pool is slightly different in size from those reported elsewhere in this supplement. These qualitative data were extracted and analyzed over a 6-week period by 2 research teams: one at UM CEPHS and one at NORC at the University of Chicago.
Both UM CEPHS and NORC separately coded the qualitative data associated with “other” responses for the occupation, certification, and program area questions in accordance with the hierarchical schema provided by the taxonomy. Researchers used additional data provided by the respondents to assist with making appropriate coding decisions. The research teams used the published version of the taxonomy to make coding decisions,18 rather than the draft version used in PH WINS, to determine the accuracy and reliability of the taxonomy in categorizing worker characteristics. Table 1 provides a crosswalk between the published taxonomy and the version used for PH WINS responses.
The qualitative data were sorted into similar themes and then into 4 groupings: (1) matched a taxonomy category in the occupation, certification, or program area axis; (2) matched a taxonomy category in another axis (eg, licensure); (3) did not match a taxonomy category but should be included in the analysis; and (4) did not match a taxonomy category and should not be included in the analysis. Occupation data were coded into 1 of the 65 categories included under the occupational axis in the taxonomy, which are grouped by Management and Leadership, Professional and Scientific, Technical and Outreach, and Support Services; credential data were coded into 1 of the 24 categories under the certification axis in the taxonomy; and the program area data were coded into 1 of the 29 categories under the program area axis in the taxonomy (eg, 1.2.4 for epidemiologist; Table 1). If a response aligned with a taxonomy category in a different axis, it was coded as such (eg, licenses reported as credentials were recoded to align with the licensure axis of the taxonomy). Responses that were not relevant to a taxonomy axis (eg, listing completed coursework as a program area) or unrelated to public health (eg, amateur radio license as a credential) were coded separately and removed from the analysis. UM CEPHS and NORC researchers compared their results specifically with regard to coding discordance and reconciled coding differences.
Finally, potential new categories within existing taxonomic axes were identified from qualitative responses that appeared frequently in the data but did not correspond to existing categories. The study design for the PH WINS analysis was reviewed by the University of Michigan institutional review board and deemed exempt from ongoing review.
Of the 23 254 PH WINS respondents who completed the question related to role in their organization (ie, occupation), 19 462 (84%) were able to categorize themselves within one of the existing taxonomy categories in that axis. “Other” was selected by 3791 (16%) respondents; 776 of these responses were excluded from the data set because they did not provide qualitative data (n = 619) or the data they provided were too vague or could not be interpreted to be recoded (n = 157). The remaining 80% (n = 3015) of “other” qualitative responses were analyzed for recoding into a taxonomy occupational category (Figure).
Twelve percent (n = 365) of the 3015 analyzable responses were dropped from the data set, leaving 2650 responses to be recoded to align with a taxonomy occupation category. Of the 365 responses dropped, 313 reflected generic job titles (eg, analyst, consultant/advisor, representative), 27 were program areas (eg, HIV, sexually transmitted disease, administration) that are not occupations, and an additional 25 were not public health occupations (eg, architects, drivers, librarians). The remaining 2650 responses were recoded into an existing category within the taxonomy axis: 44% (n = 1156) of these responses matched a specific taxonomy category, whereas 56% (n = 1494) could not be matched to a specific occupation included in the taxonomy and were instead recoded into another “other” category, including Other Professional and Scientific (n = 752), Other Technical and Outreach (n = 259), Other Business Support Services (n = 328), or Other Management and Leadership (n = 155).
Almost 70% of the 752 responses recoded as Other Professional and Scientific were represented by 6 occupation responses: specialist (n = 147), investigator (n = 109), researcher (n = 76), surveyor (n = 69), licensing or regulatory worker (n = 63), and quality assurance or improvement worker (n = 48). Nearly 90% of the responses recoded as Other Technical and Outreach included disease intervention specialist/communicable disease investigator (n = 161), data entry/abstraction worker (n = 46), and registrars (n = 19). Qualitative responses frequently identified in other categories of the occupation taxonomy included adult protective services/community worker (n = 38), which was categorized as social worker, and emergency management/emergency medical services (n = 14), which was categorized as emergency preparedness worker.
In total, 22 113 of the 23 254 responses (95%) to the PH WINS question related to occupational role were mapped to a specific category in the taxonomy occupation axis by either the survey respondent or researcher recoding.
The PH WINS question related to credentials garnered 15 719 responses, 68% (n = 10 633) of which were assigned to a taxonomy category by the respondent whereas the remaining 32% (n = 5086) selected “other.” Approximately 46% (n = 2340) of the 5086 other responses were dropped from the data set because they did not provide qualitative data (n = 836), the qualitative data provided were not analyzable (n = 70), the response was not a credential (n = 247; eg, coursework taken), or the response was not a credential related to public health (n = 1187; eg, cosmetologist, public manager, certified secretary) (Table 2).
The remaining 2746 “other” responses were analyzed for recoding. Approximately 44% (n = 1195) were mapped to a taxonomy category: 274 of the responses mapped to a category in the certification taxonomy axis, whereas 921 were recoded into categories in the licensure taxonomy axis, including registered sanitarian/environmental health specialist (n = 475), licensed clinical social worker (n = 251), RN license (n = 93), licensed practical or vocational nurse (n = 46), Doctor of Dental Surgery or Doctor of Dental Medicine license (n = 6), or other licenses not specifically captured by the taxonomy (n = 50). The remaining 1551 responses either indicated that the respondent was not formally certified (n = 74) or held a certification that could not be categorized within the taxonomy so remained coded as “other” (n = 1477). Many of these “other” responses were environmental health-related certifications, lactation educator certification, and registrar certifications, among others.
In total, 11 828 of the 15 719 responses (75%) to the PH WINS question about worker credentials could be successfully mapped to an existing taxonomy category in either the certification or licensure axes, or categorized as “not formally certified.”
Of the 20 432 respondents who answered the program area question, 78% (n = 15 957) self-selected a taxonomy program area category whereas 22% (n = 4475) chose “other.” Nearly 45% (n = 2003) of the 4475 responses were removed from the data set because they did not provide qualitative data for analysis (n = 591), or the qualitative responses were not program areas, or more information is needed to classify them (n = 1412), leaving 2472 responses for analysis. The responses that were dropped from the analysis provided varied information ranging from degrees, work locations, occupations, specialization areas, or other descriptive data that do not represent health department program areas. Of the remaining 2472 responses, 38% (n = 951) were successfully recoded into an existing program area taxonomy category whereas 62% (n = 1521) remained coded as other program area. Frequent responses in the “other” category included regulation and licensure (n = 1060), laboratory (n = 124), and informatics/information technology (n = 117).
Overall, 16 908 of 20 432 responses (83%) to the PH WINS question about program area were successfully mapped to existing categories of the taxonomy axes (Table 3).
Overall, the public health workforce taxonomy was successful in categorizing occupation, certification, and program area for the majority of PH WINS respondents, with 84%, 68%, and 78%, respectively, self-selecting an appropriate taxonomy category. After recoding qualitative data, this improved to 95% of occupation responses, 75% of certification responses, and 83% of program area responses being aligned with a public health workforce taxonomy category. These findings indicate that the taxonomy improves the level of detail with which public health workforce data may be collected, which is an achievement, given its first use in a national survey. In addition, use of the taxonomy yields improvement in reliability and accuracy of workforce data. For example, the percentage of respondents initially selecting “other” for the question related to occupation in PH WINS (16%) is lower than the percentage of respondents selecting other/uncategorized occupations in the most recent ASTHO (46%) and National Association of County & City Health Officials (21%) profile surveys2 likely due to the fact that the taxonomy provides a greater diversity of occupational categories from which respondents may choose.
The taxonomy also performed well, albeit slightly less successfully in categorizing certification and program area information than occupation information. This is not surprising because these data are not typically collected in national workforce surveys, so more refinement may be needed for these taxonomy axes than for occupation. In addition, there is greater variability associated with describing program area, as this workforce characteristic is somewhat less definitive than occupation. The findings also revealed the nebulousness around formal certification, making these responses difficult to interpret and code. Certification requirements vary widely, ranging from graduate degrees to completion of a single online module. Other certifications are regulated by an organization and periodically require continuing education, testing, and recertification. The PH WINS survey captured a mix of these types of credentials. The field still needs to determine which types of credentials are particularly important to the public health workforce, since no taxonomy can capture them all.
The analysis of qualitative PH WINS data highlighted several opportunities for refinement for the 3 taxonomy axes included in the study. For the occupation question, a fairly large percentage of the 19 462 respondents who self-selected a taxonomy category chose Other Management and Leadership, Other Professional and Scientific, Other Technical and Outreach, and Other Business Support Services. Although these are each an existing taxonomy category, it would be useful to have more information about the specific occupation of these workers. Respondents were not given the option to provide additional information under these categories, but the findings provide some examples of what some of these occupations may be. Supervisors who do not fit any of the specific categories within Management and Leadership likely chose the “other” response within this area. Regulatory staff (eg, surveyor, investigator, inspector), adult/community protective workers, and quality assurance workers are some examples of respondents who likely selected the Other Professional and Scientific category. Disease intervention specialists should be considered as a future category within Technical and Outreach based on the qualitative data provided by respondents.
The PH WINS question related to credentials, which provided the certification taxonomy categories as response choices, yielded data that were often difficult to analyze because many of the qualitative responses were not bona fide credentials. Also, it was difficult to interpret whether responses were certifications or licenses when trying to recode responses into the appropriate taxonomy category. Better data may have been obtained if PH WINS provided both certification and license taxonomy categories as response options, and would have reduced the number of “other” responses by nearly 20%.
The program area data also proved to be challenging to analyze, as many responses did not represent program areas. In addition, there may be responses that warrant consideration as an additional program area taxonomy category including licensure, certification, and regulation; laboratory; and informatics, which comprised the majority of the PH WINS “other” responses. Other refinements that could be considered include revising some of the categories to be more inclusive (eg, changing Injury to Injury and/or Violence Prevention); eliminating subcategories that often overlap (eg, combining HIV, sexually transmitted disease, and tuberculosis into a single communicable disease category); rephrasing some existing categories to be more recognizable to respondents (eg, using “chronic disease prevention and control” instead of “noncommunicable disease”); and making some subcategories into main categories (eg, “immunizations” is currently a subcategory of clinical services; many immunization workers chose “other” because they were not clinical staff).
These findings are consistent with feedback from focus groups conducted by NORC with state health department Human Resources Directors and ASTHO affiliate organizations in 2014. Focus group participants suggested adding occupation categories within the taxonomy for regulatory staff, adult protective service staff, supervisors, disease intervention specialists, and quality improvement/accreditation/planning staff, all of which were frequent responses in the PH WINS data. In addition, the focus groups suggested categories for vital records staff, communications staff, legislative staff, and health navigators, among others. For program area, in addition to the findings of this analysis, focus groups suggested the areas of credentialing, medical marijuana, school health, refugee health, chronic disease prevention, emergency medical services, minority health, and family planning (M. Meit, M. Heffernan, and K. May, unpublished data, 2015).
This study is subject to several limitations. First, the coding of responses to align with taxonomy categories is subjective. The 2 independently working research teams attempted to arrive at agreement upon which responses were eligible for recoding; however, the taxonomy category within which each response was assigned may differ somewhat between the 2 sets of analyses. This is due to varying interpretations of the responses, as well as the researchers' own experience with the structure of the public health workforce in local and state health departments, which varies by state, county, and city across the country. In addition, UM CEPHS researchers were engaged in the development of the taxonomy so they were more familiar with the tool and its intended use, which may have influenced some coding decisions; NORC researchers did not have prior experience with taxonomy development. Since each research team had similar results, we feel confident that any coding bias associated with taxonomy development was minimal.
Second, the public health workforce taxonomy provides options for survey response choices but does not provide a structure for phrasing questions. PH WINS asked respondents to identify the classification that best represents their “role” and provided taxonomy occupation categories as response choices. If the question were phrased differently (eg, choose the category that best describes your occupation), respondents may have answered differently, particularly those who provided qualitative “other” responses. The same concerns apply to the certification and program area data, as the qualitative responses provided as “other” often were not certifications or program areas at all. Some respondents may have been confused about how to respond to the question being asked using the taxonomy categories as responses. There may be value in constructing a common use question set to accompany the taxonomy to further encourage standardization of data collection and comparability of data among those using the taxonomy in their survey instruments.
Finally, both research teams encountered challenges in recoding the responses to align with taxonomy categories. For example, many respondents indicated that they were “administrators” or “coordinators.” These categories are included within the Management and Leadership section of the occupation taxonomy axis. Respondents were coded within these categories even if their responses to other questions indicated that they were not managers or leaders. The advisory committee refining the taxonomy may want to consider modifying these taxonomy categories.
The public health workforce taxonomy performed remarkably well when categorizing characteristics of the local and state public health workforce and showed substantial improvement over methods previously used in workforce surveys, although inherent difficulties remain in workforce enumeration. The varying structure and organization of local and state health departments continues to generate questions about who is considered to be a public health worker. PH WINS garnered responses from some occupations not traditionally considered being part of the public health workforce, and they were largely excluded from analysis; however, they do represent workers who have some working relationship, supplementary role, or affiliation with governmental public health. In addition, a large number of respondents classified themselves as social services staff, which may be classified as part of the health department in some states but not others. This presents a problem when determining the inclusion and exclusion criteria for public health workforce enumeration studies, although this is not a weakness of the taxonomy itself. The taxonomy improved the granularity of workforce data collected but does not provide clear criteria as to whom the categories apply. A more specific case definition for “public health worker” may address this concern in the future.
This study showed the value of the public health workforce taxonomy in improving level of detail, reliability, and accuracy of workforce data and provided several suggestions for taxonomy refinement, including the addition of, rephrasing, and consolidation of some categories. Improved workforce data could provide information needed by public health administrators to assess areas where workforce gaps or lower capacity for service delivery may exist. Use of the taxonomy in PH WINS provided a much-needed assessment on the usability of 3 axes of this schema—occupation, certification, and program area—which can be considered by the taxonomy advisory committee in its future work. Future assessments would ideally be based on surveys utilizing all 12 axes to provide a more complete picture of taxonomy successes and needed refinements.
1. Gebbie KM. The public health workforce
: key to public health infrastructure. Am J Public Health. 1999;89(5):660–661.
2. Lichtveld MY, Cioffi JP. Public health workforce
development: progress, challenges, and opportunities. J Public Health Manag Pract. 2003;9(6):443–450.
3. Beck AJ, Boulton ML. Building an effective workforce: a systematic review of public health workforce
literature. Am J Prev Med. 2012;42(5S1):S6–S16.
4. Consortium from Altarum Institute, Centers for Disease Control and Prevention, the Robert Wood Johnson Foundation, and the National Coordinating Center for Public Health Services and Systems Research. A national research agenda for public health services and systems. Am J Prev Med. 2012;42(5S1):S72–S78.
5. University of Michigan Center of Excellence in Public Health Workforce
Studies/University of Kentucky Center of Excellence in Public Health Workforce
Research and Policy. Strategies for Enumerating the U.S. Governmental Public Health Workforce
. Washington, DC: Public Health Foundation; 2012.
6. Beck AJ, Boulton ML, Coronado F. Enumeration
of the governmental public health workforce
, 2014. Am J Prev Med. 2014;47(5S3):S306–S313.
7. Sumaya CV. Enumeration
and composition of the public health workforce
: challenges and strategies. Am J Public Health. 2012;102(3):469–474.
8. Gebbie KM, Raziano A, Elliott S. Public health workforce enumeration
. Am J Public Health. 2009;99(55):786–787.
9. Gebbie K, Merrill J, Sanders L, Gebbie EN. Public health workforce enumeration
: beware the “quick fix”. J Public Health Manag Pract. 2007;13(1):72–79.
10. Tilson H, Gebbie KM. The public health workforce
. Ann Rev Public Health. 2004;25:341–356.
11. Association of State and Territorial Health Officials. Profile of State Public Health. Vol 2. Arlington, VA: Association of State and Territorial Health Officials; 2013.
12. National Association of County & City Health Officials. 2013 National Profile of Local Health Departments. Washington, DC: National Association of County & City Health Officials; 2013.
13. Boulton ML, Hadler J, Beck AJ, Ferland L, Lichtveld M. Assessment of epidemiology capacity in state health departments, 2004-2009. Public Health Rep. 2011;126(1):84–93.
14. University of Michigan Center of Excellence in Public Health Workforce
and Characterization of the Public Health Nurse Workforce: Findings of the 2012 Public Health Nurse Workforce Surveys. Ann Arbor, MI: University of Michigan Center of Excellence in Public Health Workforce
15. Centers for Disease Control and Prevention. National assessment of capacity in public health, environmental, and agricultural laboratories, United States, 2011. MMWR Morb Mortal Wkly Rep. 2013;62(9):161–164.
16. Coronado F, Polite M, Glynn MK, Massoudi MS, Sohani MM, Koo D. Characterization of the federal workforce at the Centers for Disease Control and Prevention. J Public Health Manag Pract. 2014;20(4):432–441.
17. University of Michigan Center of Excellence in Public Health Workforce
Studies. Public Health Workforce Enumeration
, 2012. Ann Arbor, MI: University of Michigan Center of Excellence in Public Health Workforce
18. Boulton ML, Beck AJ, Coronado F, et al. Public health workforce taxonomy
. Am J Prev Med. 2014;47(5S3):S314–S323.
19. Leider JP, Bharthapudi K, Pineau V, Liu L, Harper E. The methods behind PH WINS. J Public Health Manag Pract. 2015;21(suppl 6):S28–S35.