The Multi-State Learning Collaborative for Performance and Capacity Assessment or Accreditation of Public Health Departments (MLC) was a 14-month project funded by the Robert Wood Johnson Foundation and managed by the National Network of Public Health Institutes and the Public Health Leadership Society. The purpose of the MLC is to foster the continued development of existing innovative systematic performance assessment programs and widely disseminate their lessons learned in order to maximize the effectiveness of governmental public health agencies.1
Illinois, Michigan, Missouri, North Carolina, and Washington were selected from a field of 18 applicants to participate in the MLC.2 The participating states also received a grant of $150 000 from the Robert Wood Johnson Foundation to enhance their existing programs. Throughout the course of the project, the participating states formed a peer network and learning community that exchanged lessons learned about their assessment and accreditation programs with one another and with the public health practice community. With these funds, the states conducted a wide range of activities, from the development of products (eg, accreditation readiness instrument, digital library of model practices), to systematic research and planning processes (eg, social marketing research, quality improvement plans) geared to improve their existing programs.
This article discusses the specific activities the Michigan Local Public Health Accreditation Program has undertaken as a result of the support of the Robert Wood Johnson Foundation funding by defining goals, outlining outcomes, and identifying results and lessons learned in the undertaking of MLC effort.
The Michigan Local Public Health Accreditation Program3 has been in existence since 1996. It assesses the ability of local health departments to meet minimum program requirements that are based in state law, administrative rule, department policy, or professionally accepted standards of practice. Michigan's 45 local health departments are reviewed within a 3-year cycle that includes for each health department an internal self-assessment process, an on-site review, and a corrective plan of action process to correct any deficiencies found during the on-site review. The process includes the evaluation of each local health department's organizational capacity and 12 individual public health programs. A total of 122 minimum program requirements and 202 measures called indicators comprise the evaluation tool. Currently, all of Michigan's local health departments have been evaluated twice and all are accredited. In 2003, the on-site review component of the accreditation process was paused for a period of 1 year to evaluate the process and to make recommendations for improvement. A total of 44 data-driven recommendations were made by a local-state quality improvement workgroup to the Michigan Local Public Health Accreditation Commission and to the Michigan Departments of Agriculture, Environmental Quality, and Community Health.4 The funding and resources associated with the MLC allowed Michigan to implement many of these recommendations.
Michigan's MLC Enhancement Project was organized to accomplish two goals and a total of six objectives. The first goal was to enhance Michigan's program by focusing on recommendations provided by the local-state quality improvement workgroup. The second goal was to contribute to an interactive learning environment for accreditation. Michigan developed a three-tracked approach to achieve its goals. Each track had a dual purpose. Track 1 assessed Michigan's accreditation model, and developed a voluntary continuous quality improvement (CQI) component for local health departments. Track 2 developed a model to improve the interface between local health departments and reviewers, and developed tools for increasing accreditation and public health awareness of local governing entities. Track 3 established an evolving digital library of Michigan accreditation information, and developed a model for best practices exchange.
Michigan formed an 11-member steering committee to oversee the enhancement efforts. It was cochaired by a local health officer and a state public health agency administrator. The Steering Committee established three workgroups to correspond to the three tracks. The workgroups were populated with state, local, and public health institute representatives.
It was determined by the workgroup members that using the Shewhart Cycle of “Plan, Do, Check, and Act”5 CQI tool combined with the National Association of City and County Health Officials' (NACCHO's) Operational Definition of a Functional Local Health Department6 would fit well for future incorporation into a voluntary quality improvement component for Michigan's accreditation system. It was further agreed that including the National Public Health Performance Standards7 with the model could assist toward formulating outcomes measures.
Workgroup #2 members recruited state and local public health professionals with a known interest in developing a model to improve the interface between evaluators and evaluees and/or in developing the tools for increasing local governing entity awareness of accreditation and public health. Two subgroups were formed to address the elements. The first subgroup focused on improving the interface between evaluators and local health departments. The subgroup surveyed local health departments, state agencies, and other MLC members regarding their evaluator and evaluee selection and training processes. Discussions were also held to address issues involving interrater reliability.
The second subgroup focused on developing tools to increase public health and accreditation awareness of local governing entities. As with the first subgroup, other MLC states were consulted, specifically Washington and North Carolina, both of whom have functioning governing unit awareness programs.
Workgroup #3 members included a cross-section of all partners of the Michigan Local Public Health Accreditation Program, with an additional member from the Michigan State University Libraries providing further professional expertise. They also began their work in January 2006. They held a brainstorming session that encompassed myriad possibilities for knowledge management and information exchange and delivery for accreditation program topics in Michigan. The group members refined the “wish list” that was created through extensive group discussion (both real time and electronic), informal assessment of user groups, and examination of archived survey data on topics relevant to the group's charge.
In addition, the group took steps to develop an exchange of model practices rooted in Michigan's accreditation program. The group not only built on the aforementioned brainstorming and data review but also compiled and reviewed the experiences with best practices of other entities on the local, state, and federal levels. The group's consensus was that this piece should be not only a repository for what they defined as model practices but also an active forum for discussion, technical assistance, and idea exchange.
Michigan developed a discussion framework for a voluntary CQI process that is based on NACCHO's Operational Definition of a Functional Local Health Department. Figure 1 shows this process.
In the example shown in Figure 2, planning begins with selecting 1 of the 10 Essential Services8 as the goal (ie, Plan). NACCHO's Operational Definition document expands each of the 10 Essential Services with key objectives, any of which can be applied to specify what programs will accomplish in the goal for the Do step. The Check step draws from the National Public Health Performance Standards framework, using questions to ascertain which appropriate inquiry and methodology address the goals and objectives. A critical activity in the Check step is assessing the collected data.
Finally, in the Act step, the use of the Healthy People 2010 framework assists in quantifying the outcomes measures. The Act step develops important measurement and evaluation component(s) and completes the cycle, sending the process back to the Plan step.
The model is flexible because each local or state health agency can apply it, based on the assessment of what components or aspects outlined in NACCHO's Operational Definition framework (ie, goals and objectives). As shown above, employing other frameworks, especially in the Check and Act steps, is in keeping with the adaptability of the process to various health agencies' needs.
In addition, Michigan developed a set of recommendations to enhance reviewer team and local health department interface. Selection criteria, based on a combination of education, training, experience, and skills for evaluators, evaluees, and observers, were developed. In addressing interrater reliability issues, Michigan developed recommendations to address inconsistency by reducing variability in the process.
The third enhancement to the current process was to develop a model for ongoing awareness, education, and training of local government entities including an updated version of the Guide for Local Governing Entities. The guide gives an overview of public health's roles and responsibilities for both governing entities and local health officials. Additional work products include a local public health services directory, as well as an orientation manual for local public health officials to use as a toolkit for new governing entity members. The toolkit includes the revised Michigan's Guide to Public Health for Local Governing Entities, a local public health services directory template, a board member orientation manual, as well as a Roles and Responsibilities PowerPoint and a public health code reference guide that serve as supplemental resources.
Michigan also established an evolving digital library of Michigan accreditation information. The state has completed programming efforts for both the digital library and the model practice exchange. The digital library will have several different topic areas associated with accreditation. The library is scalable, which means it can be expanded to have the same level of functional utility for accreditation or performance assessment programs in other states.
In addition, the model practice exchange information network will be constructed on the existing platform of the recently expanded Michigan Local Public Health Accreditation Web site. The model for Michigan will have components that are scalable to include the broader public health practice community. Users can browse and search available practices, submit practices for consideration, download them for use, and engage in dialogue with other local health department and state agency staff to obtain technical assistance for implementation. The model practice exchange is organized by Michigan accreditation standards and uses a three-stage developmental approach to classify the practices beginning with the Prototype Phase, followed by the Promising Phase, and culminating in the Model Practice. Once the practice is viewed, downloaded, and evaluated, it is classified as promising. The practice achieves model status once it has been successfully implemented by three entities. This approach has been tested on a preliminary basis, but needs to be validated with active user involvement and further testing.
Michigan was able to meet its goals and objectives for the MLC, but it could not have accomplished as much as it did without the right mix of expertise, commitment, and resources. The consensus of the Michigan MLC Steering Committee was that much more was accomplished than originally anticipated given the tight timelines associated with the project.
As a result of participation by Michigan representatives in the Exploring Accreditation9 Project and through an ongoing interest in national initiatives and tools, the state continues to seek improvement in its program to reflect evolving public health learning about accreditation and performance assessment. Participation in the MLC contributed to additional learning in this regard. At the same time, Michigan also recognized that it needed to “raise the bar” with regard to its own accreditation program in order to continuously improve the quality of local health department performance, as all of its local health departments currently have achieved the highest designation, “Accreditation with Commendation.” Introducing the concept of a voluntary CQI component to Michigan's accreditation program will require considerable time and effort, but may be one method to further improve the quality of local public health services in Michigan.
Michigan also recognized that the issues of who should conduct reviews and interrater reliability are concerns not mutually exclusive to Michigan. During the MLC project period, the information exchanges regarding interrater reliability were numerous and productive. The state now has a national MLC network and its resources from which to draw with regard to this issue. Michigan developed evaluator and evaluee criteria that will now be reviewed by the state-local quality improvement workgroup for possible implementation.
Creating mechanisms and tools to promote an increased awareness by local governing entities of both accreditation and local public health was identified as 1 of the 44 data-driven quality improvement recommendations by the state-local workgroup in 2004. The toolkit developed by Michigan has been shared with local health officials, and anecdotal reports indicate that it is being used to educate and orient boards of health, county commissions, as well as state and federal legislators.
The digital library and model practice exchange have also implemented recommendations from the state-local quality improvement workgroup. It will still be several months before the success of these two products can be measured. However, preliminary reports based on the user feedback forms received seem to indicate that both products will be value added to the Michigan Local Public Health Accreditation Program and may be scaleable and replicable in other states.
Michigan will continue to meet with key stakeholder groups such as the Michigan Association for Local Public Health and the Michigan Public Health Accreditation Commission to further enhance the work products of MLC-1. It will also be able, ascribable to the partnerships that have been enhanced and strengthened as a result of MLC-1, to work collectively to “raise the bar” in Michigan in order to further improve the performance of local public health in hopes of improving health outcomes in the future.