Davis, Mary V. DrPH, MSPH; Reed, Joy EdD, RN; Devlin, Leah M. DDS, MPH; Michalak, Craig L. MBA; Stevens, Rachel EdD, RN; Baker, Edward MD, MPH, MSc
Mary V. Davis, DrPH, MSPH, is Director of Evaluation Services, North Carolina Institute for Public Health, University of North Carolina at Chapel Hill School of Public Health.
Joy Reed, EdD, RN, is Branch Head, Local Technical Assistance and Training, and Head of Public Health Nursing, Division of Public Health, North Carolina Department of Health and Human Services, Raleigh.
Leah M. Devlin, DDS, MPH, is State Health Director, Division of Public Health, North Carolina Department of Health and Human Services, Raleigh.
Craig L. Michalak, MBA, is the Accreditation Administrator, North Carolina Local Health Department Accreditation Board, and Program Officer for the North Carolina Institute for Public Health and the Public Health Leadership Program, School of Public Health, University of North Carolina at Chapel Hill.
Rachel Stevens, EdD, RN, is Senior Advisor, North Carolina Institute for Public Health, University of North Carolina at Chapel Hill School of Public Health.
Edward Baker, MD, MPH, MSc, is Director, North Carolina Institute for Public Health, University of North Carolina at Chapel Hill School of Public Health.
Corresponding author: Mary V. Davis, DrPH, MSPH, North Carolina Institute for Public Health, School of Public Health, University of North Carolina, Chapel Hill, NC 27599 (firstname.lastname@example.org; email@example.com).
North Carolina is one of several states that have developed an accreditation system for local public health agencies. The purpose of the North Carolina Local Health Department Accreditation (NCLHDA) system is to ensure that all local public health agencies have the capacity to provide a standard set of essential services on a statewide basis.1 This system was created and implemented by the NC Association of Local Health Directors, the NC Division of Public Health, and the NC Institute for Public Health at the University of North Carolina School of Public Health. These organizations formed the NC Accreditation Learning Collaborative (NC Collaborative) to improve and enhance the NC accreditation system and to participate in the Multi-State Learning Collaborative to advance public health agency performance and capacity assessment efforts. This article presents NC Collaborative projects that have potential implications for the proposed national public health accreditation model.2
The full history of the NCLHDA has been described previously.3 Briefly, NCLHDA development occurred in three phases: (1) the NC Association of Local Health Directors initiated the process and explored accreditation models and established the basic process with the NC Division of Public Health and the NC Institute for Public Health; (2) the partners conducted two pilots of the system; and (3) the partners fully implemented the NCLHDA. In 2005, the NC legislature passed Senate Bill 804, which created and funded the permanent system. Permanent Rules were enacted in October 2006. Currently, NCLHDA is a mandatory system with standards approved by the independent NCLHDA Board, the NC Commission for Health Services, and the NC Rules Comission.4
As of January 2007, 25 of the 85 NC local health departments have been accredited. Thirty additional health departments have volunteered and have been scheduled to undergo accreditation by 2009. The current system has the following components: training for site visitors, agency staff, and NC Division of Public Health consultants; technical assistance for local health departments; an agency self-assessment; site visit by a team of four visitors to amplify, verify, and clarify the agency self-assessment; a site visit report to the Accreditation Board; action by the Accreditation Board; an appeals process; corrective action plans as needed; and comprehensive evaluation for system monitoring and quality improvement.
The NC Collaborative conducted several initiatives to enhance the NCLHDA and contribute to the Multi-State Learning Collaborative. Three projects, an improvement fund for accredited health departments, improvement of specific NCLHDA standards, and examination of the effectiveness of the NC Division of Public Health technical assistance consultants, enhanced the NCLHDA program. Two initiatives, costs and benefits of accreditation and an Accreditation Road Map, may inform the next steps of the proposed national accreditation model and are presented in detail below.
Costs and benefits of accreditation
To understand agency costs to prepare for and undergo accreditation, NC Institute for Public Health, as Accreditation Administrator, surveyed the 10 local health departments undergoing accreditation during the FY 2006 cycle. Survey items included staff time in hours spent on specific tasks to prepare for accreditation, completing the agency self-assessment instrument, preparing for the site visit, coordinating the site visit, and on tasks after the site visit.
During the two NCLHDA pilots, health directors, agency personnel, site visitors, and consultants provided anecdotal evidence of benefits they gained from the accreditation process. NC Collaborative partners reviewed these benefits, identified others, and organized these into the list of benefits, as given in Table 1.
The final list of benefits was added to existing NCLHDA evaluation protocols. The NCLHDA system includes a comprehensive evaluation in which all program participants are given an opportunity to provide feedback about the implementation of the system during a given accreditation cycle. Surveys are conducted with a primary staff contact at local health departments undergoing accreditation, site visitors, and NC Division of Public Health consultants. Interviews are conducted with health directors of agencies undergoing accreditation and NC Collaborative staff.
In the NCLHDA FY 2006 cycle, evaluation survey and interview items asked participants to indicate which benefits they personally experienced or observed that others experienced. They were also given an opportunity to identify other benefits not listed. Benefits items were tailored to participant categories (eg, agency personnel, health directors, site visitors). Participants were asked only about benefits that they could experience or observe. Ten benefits, identified in the table, were included in all surveys and interviews.
Accreditation Road Map
The purpose of the Accreditation Road Map is to provide public health partners with a checklist of issues to consider when thinking about creating a new accreditation system for local and/or state public health agencies. NC Collaborative partners and two national consultants identified five phases of creating and implementing an accreditation program using the Turning Point Performance Management Self-Assessment Tool5 as a model framework.
Costs and benefits
The 10 health departments that went through accreditation in FY 2006 submitted reports on staff time effort on the identified tasks. These data varied considerably among health departments, making it difficult to summarize or draw conclusions from these data.
Seventy-seven individuals participated in the evaluation process as follows. All 10 local health department primary contact staff completed surveys. Twenty-three of 27 (85%) site visitors completed the site visitor survey. All 5 NC Division of Public Health consultants with lead responsibilities for accreditation and 25 programmatic consultants completed the consultant survey. All 10 local health department directors in agencies that went through accreditation completed interviews and 4 staff at NC Collaborative partner organizations completed interviews.
Among the 10 common benefits, 65% or more of all participants observed or experienced the following: “improved understanding of public health,” “creating or updating policies,” “highlighted health department strengths,” and “identified areas for improvement.” Respondents in various participant groups (eg, local health department staff, site visitors) differentially experienced or observed benefits. For example, among local health department staff, all 10 rated “creating or updating policies” and 9 of 10 rated “improved understanding of public health,” “highlighted health department strengths,” “active community and agency partnerships,” and “identified areas for health department improvement” as benefits. The 10 health directors interviewed indicated that they experienced 7 of the 10 common benefits. Nine health directors also indicated that benefits of accreditation included “identifying procedures that are already working well” and “health department staff are valued by other agencies.” Site visitors' experience with or observation of the 10 common benefits ranged from 40% (“motivated staff”) to 80% (“creating or updating policies”). Ninety percent of site visitors, however, reported experiencing the ability to apply what they learned from accreditation in their own health departments as a benefit. NC Collaborative partners were more likely to indicate that they observed all 19 benefits included in their structured interviews. For 13 benefits, these partners reported observing these benefits; among the remaining 6 benefits, 3 of the 4 partners observed these benefits. NC Division of Public Health consultants' responses varied according to consultant type. Lead consultants (n = 5) generally reported observing all benefits, and the programmatic consultants that responded to this question (n = 12) reported observing fewer benefits.
Accreditation Road Map
The phases and major considerations for the Accreditation Road Map are provided in Table 2. The full road map can be found at http://www2.sph.unc.edu/nciph/accred/index.htm. During the Multi-State Learning Collaborative final meeting, September 20–22, 2006, the NC Collaborative presented the Accreditation Road Map to representatives from other states and staff from the National Association of County and City Health Officials, the Association of State and Territorial Health Officials, and the Robert Wood Johnson Foundation. As can be seen in the table, the road map emphasizes partnership and communication as much as, or even more than, actually creating and implementing an accreditation program. This emphasis reflects the experience of NC Collaborative partners as well as other Multi-State Learning Collaborative states. Meeting participants supported the phases and elements as presented. Participants identified the road map as a strategic document rather than a checklist, and suggested that critical steps or elements that will gain traction for an accreditation program be identified. They recognized the importance of each road map phase, but indicated that the development of their particular system did not necessarily follow the phases in the order presented. They also emphasized that elements and phases are recurring processes, and that an accreditation system “doesn't stay done.”
The NC Collaborative participation in the Multi-State Learning Collaborative enhanced the NCLHDA by reinforcing the continuous quality improvement focus, providing an opportunity to step back and review processes, and supporting specific process improvements. Participation in Multi-State Learning Collaborative meetings and conference calls introduced NC Collaborative partners to colleagues in four other states working on similar issues. Our exchanges of ideas, practices, and policies informed NCLHDA process improvements and, hopefully, informed the national process.
The accreditation benefits and Accreditation Road Map projects have sparked interest by national organizations involved in the Exploring Accreditation process that provided recommendations for creating a voluntary national public health accreditation model.4 Benefits and incentives of accreditation was one of four focus areas in this process. According to Mays,6 accreditation programs with strong and visible incentives that encourage organizations to undergo accreditation are more likely to function successfully. Benefits and incentives can balance agency costs to prepare for and participate in accreditation systems. Previously identified incentives for accreditation include targeted funding for accredited agencies, funding levels that reward accreditation, professional recognition or validation of accredited agencies, and access to resources that encourage performance improvement.2,6 Identifying and communicating accreditation benefits encourages support for the system by the various parties involved (site visitors, agency personnel, etc), encourages health officials to volunteer their agencies for accreditation, and provides data that enable local officials, such as county commissioners, to support local health department accreditation activities.
Results from the NCLHDA 2006 evaluation findings confirm that accreditation participants experience a number of benefits and that benefits vary among different types of system participants. Further data analysis will focus on identifying which benefits could be incentives for different groups, such as health officials and county commissioners. For example, which benefits could be the most important incentives to encourage health directors to participate in accreditation? Which benefits are the most important incentives to encourage local officials, such as county commissioners, to provide funds to health departments to prepare for accreditation? In addition, it will be important to create communication strategies about the incentives targeting these various groups. Answers to these questions will contribute to the next steps in creating the national public health accreditation model.
The Accreditation Road Map may also be valuable to national organizations that move forward with the Exploring Accreditation recommendations. Establishing an accreditation program requires more than creating an agency self-assessment instrument and developing a process. It requires support from a variety of stakeholders, such as local health departments, the state health officer, and political leaders at the state and local levels. The proposed national model does not preclude states and partnerships from creating accreditation programs. In addition, within each state, support for and agreement on the purpose of accreditation programs can ensure that a model will be adopted by local agencies. Furthermore, understanding which components of the Accreditation Road Map are essential to increase successful adoption of accreditation systems may inform the public health accreditation research agenda.
3. Beitsch LM, Thielen L, Mays G, et al. The Multi-State Learning Collaborative, states as laboratories: informing the national public health accreditation dialogue. J Public Health Manag Pract. 2006;12(3):217–231.
5. Turning Point Performance Management National Excellence Collaborative. Performance Management Self-Assessment Tool. Published 2004. http://www.phf.org/PMC.htm
. Accessed February 12, 2006.
6. Mays G. Can Accreditation Work in Public Health? Lessons Learned From Other Service Industries. University of Arkansas College of Public Health; November 30, 2004. White paper for the Robert Wood Johnson Foundation.
© 2007 Lippincott Williams & Wilkins, Inc.