Morrow, Cynthia B. MD, MPH; Nguyen, Quoc V. MD; Shultz, Rebecca G. MPH; Murphy, Jill M. PhD; Mignano, Michelle A. MPA
Over the past few years, there has been growing interest in public health quality improvement (QI), as evidenced by the recent publication of consensus definitions of “quality in public health” and QI.1,2 Furthermore, participation in the voluntary national accreditation process has the potential to improve the public's health and transform local health departments (LHDs) into high-performing organizations that value continuous QI. Drawing from over 10 years of experience in formal QI activities, this article presents an overview of the QI process at a mid-sized LHD. It is intended to provide a possible framework for other LHDs that are considering establishing a formal QI plan or expanding their current QI plan.
According to the 2008 National Profile of Local Health Departments Study, more than half of LHDs surveyed intended to seek voluntary national accreditation.3 The LHDs indicated that the promotion of high performance and continuous QI were among the top benefits for those seeking accreditation.3 The standards and measures set forth by the Public Health Accreditation Board include 1 domain specifically addressing the development and implementation of QI in public health processes, programs, and interventions, while also infusing QI across all domains.4
For an LHD to move toward accreditation, it will be imperative to establish and implement a QI process within the agency; yet, only half of the LHDs surveyed in 2008 reported performing any formal QI activities. Mid-sized and large LHDs were disproportionately involved in QI activities.4 There are many challenges faced by LHDs seeking to institute a QI program, including prioritization of limited resources. This article addresses some of the challenges faced by an LHD, as it transforms its organizational culture to embrace QI as a means to improve its services and the community's health.
The Onondaga County Health Department (OCHD), located in Syracuse, New York, is a full-service LHD that serves more than 467 000 residents in central New York.5 (Refer to Figure for an organizational chart for OCHD.) In 2011, OCHD employed 280 full-time and 20 part-time employees and had an adopted budget of $79 million, of which $26 million (including $10 million in grants) were for core public health activities.
FIGURE . Onondaga Co...Image Tools
The institutional review board was consulted, and it was determined that this study did not need to be reviewed, because it did not meet institutional review board's definition of “Human Subject Research.”
A History of Quality Assurance and Improvement Efforts
In 1989, OCHD founded a quality assurance (QA) committee, which was charged with reviewing issues pertaining to aspects of New York State's Public Health Law. This committee primarily focused on reviews to ensure that the clinical functions of the LHD complied with regulations concerning the use of pharmacologic agents or the accreditation of health care workers.
Quality improvement: the path to the summit
In 1998, the leadership of OCHD convened the first annual QA summit in which programs presented their main QA projects. Within a few years, the focus shifted from solely QA to both QA and QI, as there was increasing recognition of the need to improve certain problematic areas and to become more responsive to client satisfaction. By 2000, the first annual QI summit was held. The intent of the QI summit was to encourage OCHD program directors across the department to develop, implement, evaluate, and present the results of focused interventions to improve programs' effectiveness or efficiency. While no formal tools for QI were required to be used, a 4-step format involving collection of baseline data, implementation of a specific intervention, postintervention data collection, and analysis of pre- and postintervention data was emphasized to staff involved in QI projects. In retrospect, these early QI efforts were representative of “Small qi” by Riley et al6 (small, incremental improvements), but they did not yet reflect the organization's cultural shift to the large, meaningful changes or “Big QI.”6(p74)
To ensure the success of the summit, a Quality Improvement Methodology Committee (QIMC) was established to provide support to program directors. The QIMC includes OCHD staff members, trained in QI methods and processes, who provide the framework and the support system necessary for the endeavor. The program directors are responsible for selecting and proposing a potential project for the QI summit. The QIMC then reviews each project to ensure that it has a sound design, a plausible intervention, and measurable outcomes that are amenable to statistical analysis. An emphasis is placed on the relevance and significance of the project's potential impact on the public's health. After projects meeting these criteria are identified, the QIMC presents the proposed project list to the leadership of the OCHD for further feedback and final approval. Once a project is approved, a QIMC member provides ongoing guidance to the project staff on study design, data collection, statistical methods, presentation aids, and any other necessary supports. The QIMC convenes monthly throughout the project period to report on progress and to identify and mitigate any problems.
In preparation for the QI summit, each project team develops a PowerPoint presentation summarizing the project and prepares to deliver the presentation. The QI summit is organized as a day-long event held at an off-site location to showcase the projects. All health department staff, local leaders from County Executive's Office, the health advisory board, and the legislature are invited. On average, approximately 50 individuals attend the summit every year.
A brief review of more than a decade of QI summits
Between 1998 and 2010, 212 individual projects were presented at the annual summits. Table 1 provides a broad overview of the distribution of projects by the Division or Bureau. For the first 3 summits, project involvement was limited to core public health programs. However, in 2000, the leadership of OCHD recognized the importance of having administration participate as well. Since then, staff members from the fiscal office, information technology, and other administrative support services have implemented projects with the understanding that management practice directly impacts program performance.
Further descriptions of projects from the 2010 summit, including their findings and their potential impact, are presented in Table 2. The brief project descriptions illustrate the broad range of projects selected. In an effort to encourage broad staff participation in the QI summit, some projects approved for presentation may not have involved the improvement of an existing process or service.
Over the years, cross-cutting projects involving collaboration of different programs have been encouraged as a means of promoting coordination of services among programs targeting the same population. These collaborative efforts result in improved efficiency and shared expertise. As an example, in 2005, the Lead program teamed with the Early Intervention (EI) program to implement an intervention designed to increase lead testing rates among EI participants. It has also become evident over time that some projects require longer data collection or intervention periods to produce more meaningful results. For this reason, multiyear projects have been encouraged when appropriate and feasible. For example, in 2009, the fiscal office implemented a new procedure for processing contract requisitions. The postintervention data collection required waiting for enough contracts to be initiated or renewed to accurately assess the new procedure. The baseline data and intervention were presented at the 2009 summit, while results were presented in 2010. Such scalable projects provide more opportunities for program directors to implement meaningful projects.
Occasionally, when sufficient resources are available, an LHD can adopt a more robust approach to QI. In 2010, OCHD was awarded a grant to improve perinatal outcomes in Medicaid recipients. With this grant support, a partnership was formed with Shaffer Consulting to utilize a “WorkOut” process to identify, implement, and measure specific strategies to improve OCHD's effectiveness in this area. In 100 days, referrals to home-visitation services dramatically increased and those gains have been sustained. (Refer to Text box 1 for additional details about the “WorkOut.”)
TEXT BOX 1 A Differ...Image Tools
While the QI summit has been held annually since 2000, formal evaluation of the QI summit did not occur until recently. In 2009 and 2010, an online survey was administered to staff following the QI summit. The purpose of the survey was to measure satisfaction with the QIMC, assess the perceived importance of the QI process, and solicit feedback on ways to improve future experiences. More than 100 people attended the summit in either 2009 or 2010. Of these, 57% responded to the online survey. The results are limited by a poor response rate, but 80% of respondents who were directly involved in the QI summit projects reported that they agreed or strongly agreed that the QI process was important both to their bureaus and to the health department's mission. In addition, 16 of the 17 attendees who were not directly involved in QI projects agreed or strongly agreed with the statement “The QI process is important to the Health Department's mission.” Feedback from open-ended survey questions on how to improve the QI summit process, such as obtaining earlier approval of projects and moving the summit to a less-active time of the year, resulted in changes that were recently implemented.
The transformation of “small qi” into “big QI”
Recently, Riley and Brewer7 introduced the concept of “Small qi” and “Big QI,” with the former representing the “actual conduct of improving a process at the microsystem level using an integrated set of QI methods and techniques...” and the latter, “complete organizational involvement.”7 While the first few years of QI efforts were centered on an annual presentation of QI projects at a summit, more recently, OCHD has begun to shift from an organizational culture emphasizing “Small qi” to the one addressing “Big QI.” “Small qi” continues to anchor QI efforts at the OCHD. However, changes in the leadership's priorities with growing recognition of the critical importance of QI in strategic planning, in preparing for accreditation and in addressing personnel policies, have elevated the relative importance of QI throughout the department.
In 2002, in recognition of the growing emphasis on QI, the QA Committee was changed to a QI/QA Committee. Staffed by senior representatives from most OCHD programs, the committee meets at least 10 times a year. The committee is charged with addressing both QA and QI issues, which are inherent in all functions of OCHD. Its scope includes timely evaluation of core services, review of policies, adoption of new and improved ways of offering services, and examination of errors and evaluation of remedial actions. Because each item/topic is presented to a cross section of bureau representatives, this process naturally fosters collaboration across areas of the OCHD. The majority of activities of this committee are still QA; however, the venue also provides an opportunity to present small-scale QI projects that are important to a program's services but do not carry enough weight to be considered for the QI summit.
Another example of efforts to expand the culture of QI throughout the department occurred in fall 2010. At that time, individual performance measures were introduced as part of the annual performance evaluations for every public health care worker on staff. The performance measures are intended to be closely related to program impact. Previously, only certain staff members had performance measures, which were required by outside entities (eg, Early Intervention program or Special Supplemental Nutrition Program for Women, Infants, and Children). To ensure consistency across the department, every employee of the OCHD will have at least 2 performance measures integrated into their annual performance evaluations by the end of 2011. When indicated by the measures, targeted support will be provided to ensure that public health care workers are successful in meeting their specific measures.
The degree to which public health agencies can ensure that QI is achievable in their organization is heavily dependent on their ability to establish and sustain a strong QI process. More than 10 years ago, OCHD held its first formal annual summit. Since that time, the QI process at this mid-sized LHD has been refined to introduce a more rigorous QI methodology, accommodate more collaboration between programs, increase the impact of the projects, and promote integration of QI across the department. Many factors have contributed to the success of the QI process, although barriers to its full integration still remain.
Several authors have addressed the necessary components of, and some of the barriers to, the success of QI in a health department.6–9 The following factors contributed to successes in integrating QI at OCHD:
* A top-down approach with strong support and vision to devote resources to the QI summit. The core administrative team and the senior staff members are encouraged to promote the culture of QI in routine activities. Over the years, several local legislators and officials from the County Executive's Office have attended the summit, sending a clear message to staff about the importance of QI.
* A bottom-up approach to “Small qi” projects to encourage the participation of all levels of staff. The QI summit projects are initiated and developed by program staff to ensure buy-in. Furthermore, staff members from every program participate in reviews through the QA/QI committee.
* Support for a broad range of QI activities, including small monthly projects for the QA/QI committee, projects for the QI summit, and grant-supported more elaborate QI projects. This flexibility accommodates different program and staff needs and allows for a gradual introduction to QI for staff members who may feel intimidated by large QI projects.
* Culminating annual presentations that provide an opportunity for staff to learn about and learn from other programs within OCHD, while showcasing the impact that QI has on the organization.
* A QIMC with skills and training in QI methodologies to provide guidance and support to program staff.
Despite the previous factors, OCHD staff members have identified the following as barriers to the implementation of “Big QI”:
* Inadequate resources necessary to ensure full integration. The most significant resource required is staff time. With LHDs across the country experiencing significant budget cuts, providing dedicated time for staff to carry out projects and attending trainings are increasingly challenging. According to the National Association of County and City Health Officials, 44% of LHDs reported that they had lost staff between 2008 and 2010, resulting in an approximate 19% reduction in the nationwide LHD workforce.10
* Inconsistent buy-in: While the majority of senior staff members welcome the QI experience, some individuals still feel that QI is not important to their daily work. These individuals may undermine the QI process. Securing total buy-in from frontline workers has been challenging as some may view QI projects as an added burden to their already busy day.
* Suboptimal QI summit participation: All health department employees are invited by the leadership to attend the summit; however, very few do so unless they are directly involved in a project. This may reflect the frontline staff's lack of recognition of the value of QI and/or supervisors' lack of support for their program staff to take time away from daily work duties to attend the summit.
Future directions for OCHD
While OCHD continues its efforts with “Small qi,” further changes still need to occur before OCHD can consider its transformation to “Big QI” complete. This article describes the efforts this LHD has taken to foster a culture of QI across all aspects of its functions as well as to prepare for accreditation. To achieve this transformation, OCHD will take the following steps:
* Ensure that QI is integrated into strategic plans.
* Increase training opportunities to expand the program staff's knowledge base of QI methodologies. Training sessions on topics such as the essential role of QI, tips for how to select a meaningful project, established QI tools, basic data analysis, and hints for an effective presentation will be open to all OCHD staff. This is an effort to increase the staff's comfort level with the QI process and improve the quality of future projects.
* Ensure that all future projects selected for the QI summit use specific tools, such as Deming's Plan-Do-Check-Act tool, to enhance the rigor of the projects, with greater emphasis on multidimensional interventions.11 Moving forward, the bar will be raised to make certain that all projects presented at the summit involve staff buy-in, implementation of a specific QI tool, significant impact to the program, and more robust interventions.
* Ensure that future QI projects are linked to the County's Community Health Assessment, performance measures, or other sources of data.
* Adopt a stricter evaluation of both the QI summit and “Small qi” projects to ensure the utilization of QI tools and selection of appropriate QI initiatives. The results of such evaluations can inform OCHD leadership and QIMC in ongoing efforts to improve the QI process.
* Seek grant funding for QI activities, including providing opportunities for training of key staff and for increasing staff participation (potentially through incentives, awards, or other means of recognition). The OCHD has a collaborative relationship with both SUNY Upstate Medical University and with Syracuse University, which will enhance the opportunity for training and grant funding to further QI efforts.
“Quality in public health is the degree to which policies, programs, services, and research for the population increase desired health outcomes and conditions in which the population can be healthy.”1 There are many reasons for LHDs to move toward comprehensive agency-wide QI. As LHDs gain experience in the implementation of QI, either through the accreditation process or through self-initiated efforts, the evidence base for successful QI programs will follow. This article characterizes the evolution of QI efforts at a mid-sized LHD over more than a decade. Although barriers to the full implementation of the QI process such as suboptimal and inconsistent staff buy-in and limited resources persist, OCHD has found success by combining a top-down and bottom-up approach to encourage staff involvement, supporting a broad range of scalable projects, hosting a culminating annual summit event, and forming a committee to support QI activities. Other LHDs may consider this OCHD case study as they embark upon their own QI journey.
1. Honoré P, Wright D, Berwick D, et al. Creating a framework for getting quality into the public health system. Health Aff. 2011;30(4):737–745.
2. Riley WJ, Moran JW, Corso LC, Beitsch LM, Bialek R, Cofsky A. Defining quality improvement in public health. J Public Health Manag Pract. 2010;16(1):5–7.
3. Beitsch LM, Leep C, Shah G, Brook RG, Pestronk RM. Quality improvement in local health departments: results of the NACCHO 2008 survey. J Public Health Manag Pract. 2010;16(1):49–54.
6. Riley W, Parsons HM, Duffy GL, Moran JW, Henry B. Realizing transformational change through quality improvement in public health. J Public Health Manag Pract. 2010;16(1):72–78.
7. Riley W, Brewer R. Review and analysis of quality improvement techniques in police departments: application for public health. J Public Health Manag Pract. 2009;15(2):139–149.
8. Gunzenhauser JD, Eggena ZP, Fielding JE, Smith KN, Jacobson DM, Bazini-Barakat N. The quality improvement experience in a high-performing local health department: Los Angeles County. J Public Health Manag Pract. 2010;16(1):39–48.
9. Leep C, Beitsch LM, Gorenflo G, Solomon J, Brooks RG. Quality improvement in local health departments: progress, pitfalls, and potential. J Public Health Manag Pract. 2009;15(6):494–502.
local health departments; quality improvement
© 2012 Lippincott Williams & Wilkins, Inc.