Madamala, Kusuma PhD, MPH; Sellers, Katie DrPH, CPH; Beitsch, Leslie M. MD, JD; Pearsol, Jim MEd; Jarris, Paul MD, MBA
Recent national attention has focused on improving the performance of the US governmental public health system.1–3 Since the previous data profile of state and territorial health agencies in 2007, the number of national events, programs, and publications concentrating on driving quality improvement (QI) in health departments has continued to grow.4 Moreover, public health simultaneously faces numerous challenges and opportunities that ultimately will determine the depth and extent of QI uptake.
Dramatic budget battles have been fought in each state, often mirroring the titanic struggle at the federal level. Almost every state and most local health departments have struggled with major funding cutbacks and substantial layoffs of key personnel.5–7 This has occurred as a consequence of the most severe economic downturn in US history since the Great Depression, even as demand for public health services has surged. The dichotomy between demand and capacity has been further exacerbated by intermittent public health crises, like H1N1, requiring the focused attention of the entire workforce for long stretches of time.
On the opportunity side of the equation, a great deal of national momentum has been achieved toward a national voluntary accreditation program, which has recently finalized its first set of accreditation standards following a successful beta test with 30 state, local, and tribal health departments.8,9 The foundation for accreditation and its standards has been pivotal historical antecedents: state accreditation programs in Michigan, Missouri, and North Carolina,10–12 and national standards for state and local health departments and their governing bodies, developed via the National Public Health Performance Standards Program.13 Demonstrations such as the Multi-State Learning Collaborative have shown that QI and accreditation are sound strategies for strengthening health department performance.14 The enactment of national health reform also earmarked resources for each state to establish or expand its QI capacity through the National Public Health Improvement Initiative (NPHII), aligning federal funding support with key infrastructure objectives of Healthy People 2020.15–17 The recent release of county health rankings for communities in each state across the country provided further impetus for public health action.18 Re-enforcement of QI as a critical strategy has come in the form of a recent Department of Health and Human Services Report delineating “Priority Areas for the Improvement of Quality in Public Health.”19 To aide practitioners a definition of QI in public health has also been developed and published.20
Performance management, community assessment, health improvement planning, and QI by State Public Health Agencies (SPHAs) were last assessed in 2007.21 The purpose of this study was to update our understanding of these activities in the context of events swirling around the larger public health community, and explore interest in accreditation by State and Territorial Public Health Agencies.
The Association of State and Territorial Health Officials (ASTHO) conducted its Web-based profile survey, the second national profile of SPHAs, between April and November 2010. The purpose of the survey was to document the structure, functions, and resources of state health agencies. Iterative draft versions of the survey were reviewed by SPHA senior staff, ASTHO Alumni (former state health officials), public health systems researchers, ASTHO staff, and representatives of other national public health organizations. Peer networks convened by ASTHO, including the ASTHO chief fiscal officers workgroup and the performance improvement network (a network of state health agency performance improvement officers), contributed to the development of different sections of the survey. The survey was then pilot tested by a select group of public health experts including senior staff from state health agencies and partner organizations. The final surveys were sent to senior deputies in the 59 State and Territorial Health Agencies (50 states, 8 territories and freely associated states, and the District of Columbia). Telephone and e-mail follow-up were conducted as needed. Organizational leaders at each SPHA completed sections of the 152-item questionnaire. The portion of the survey that focused on planning and QI was to be completed by the performance improvement officer or equivalent. The information collected describes agencies rather than individuals and was, therefore, exempt from human participants' protections.
There were 3 specific objectives of this study. The first objective was to examine the progress of state/territorial health assessment, health improvement planning, performance management (PM), and QI activities at state/territorial health agencies and compare findings to the 2007 findings when available. A second objective was to examine respondent interest and readiness for national voluntary accreditation. A final objective was to explore organizational factors (eg, leadership and capacity) that may influence QI or accreditation readiness.
Tabulated percentages facilitated the comparison between the 2007 data and 2010 data when available. To assess the first study objective, descriptive statistics were performed of health assessment, health improvement planning, strategic planning, PM, and QI at state and territorial health departments and compared to the 2007 data when available. Results do not reflect PM or QI activities at the county or city level. Respondents noted if and when they had developed a state health improvement plan (SHIP). A “health improvement plan” was defined in the survey as “a series of timely and meaningful action steps that define and direct the distribution of services, programs, and resources to improve your state/territory's health, or define strategic action steps to improve health status in the state/territory.” The definition of a SHIP remained unchanged from 2007.
Respondents also noted if they had a performance management and/or a QI process. If they had a performance management or a QI process, they could describe each process as partially implemented for specific programs, fully implemented for specific programs, partially implemented department-wide, and/or fully implemented department-wide. All definitions for the performance management process were derived from the Turning Point Performance Management Collaborative22 and remained unchanged from 2007. A performance management process was defined in the survey as “including performance standards, performance measures, reporting of progress and a QI process.” The definition of performance standards provided in the survey was “objective standards or guidelines that are used to assess an organization's performance; standards may be set by benchmarking against similar organizations, or based on national, state or scientific guidelines.” Performance measures were defined as “any quantitative measures or indicators of capacities, processes, or outcomes relevant to the assessment of an established performance goal or objective.” Reporting of progress was defined as “documentation and reporting of progress in meeting standards and targets and sharing such information through feedback” to the agency (or SPHA). The definition for QI was revised from 2007. The definition provided in the 2007 survey was “the establishment of a program or process to manage change and achieve QI in public health policies, programs or infrastructure based on performance standards, measurements, and reports.” The definition provided in the 2010 survey was “a formal, systematic approach (such as Plan-Do-Check-Act) applied to the processes underlying public health programs and services to achieve measurable improvements.”20
To assess the second and third objectives, descriptive analyses and cross tabulations were conducted using variables describing accreditation, leadership, and capacity. Analyses were performed using SPSS version 19.23
By November 30, 2010, all 50 states, the District of Columbia, and 2 territories had responded to the survey, yielding a final response rate of 100% of states and District of Columbia, and 93% overall.
State/territorial health assessment
Respondents were asked in 2010 if their agency had conducted a state/territorial health assessment. Forty-seven percent (n = 24) of respondents completed an assessment within the last 3 years, 8% (n = 4) developed an assessment more than 3 years ago but less than 5 years ago, 10% (n = 5) developed an assessment 5 or more years ago, 10% (n = 5) had not developed a health assessment, but plan to develop in the next year and 26% (n = 13) had not developed a state/territorial health assessment. Development of a state/territorial health assessment was not asked in 2007.
State health improvement plan
In 2010, 37% (n = 19) of respondents developed a SHIP within the last 3 years compared with approximately 24% (n = 12) in 2007. Approximately 24% (n = 12) of respondents in 2010 developed their plan more than 3 years ago compared with 57% (n = 29) in 2007. Approximately 26% (n = 13) of respondents reported they have not developed a SHIP in 2010 compared with 20% (n = 10) of respondents in 2007. In 2010, approximately 14% of respondents (n = 7) stated that they do not currently have a SHIP but plan to develop one in the next year. The response option of planning to develop a SHIP in the next year was not provided in the 2007 survey.
Of respondents who had developed a state/territorial health improvement plan, approximately 77% (n = 23) indicated in 2010 that they were planning to update the plan within the next 3 years compared with 95% (n = 38) in 2007. In 2010, 36% (n = 18) of states developed the plan using the results of the state/territorial health assessment. In 2007, 68.2% (n = 30) of respondents developed the plan using the results of the state/territorial health assessment. The number of respondents who reported linkage of the SHIP to local health improvement plans decreased slightly from 2007 to 2010. In 2010, 16% (n = 8) of respondents reported linkage to all local plans and 54% (n = 27) of respondents reported linkage to some local health improvement plans. In 2007, 29.5% (n = 13) of respondents reported that the SHIP was linked to all local health improvement plans and 29.5% (n = 13) of respondents reported linkage to some plans.
Two additional respondents (n = 40, 85.1%) had developed an agency-wide strategic plan in 2010 compared with 2007 (n = 38, 76%). When asked the status of the plan in 2010, 13% (n = 6) of respondents had not yet implemented it, 6.5% (n = 3) of respondents had implemented the strategic plan in the past year, 37% (n = 17) of respondents reported implementing the plan more than 1 year ago, but indicated that there had been no written evaluation on progress toward strategic goals, objectives, or targets, and 28.3% (n = 13) of respondents reported implementing the strategic plan more than 1 year ago, and that they had 1 or more completed written evaluations on progress toward strategic plan goals, objectives, or targets.
The 2010 survey assessed the specific timeframe during which the strategic plan was developed. Fifty-seven percent (n = 27) of respondents developed the strategic plan in the last 3 years, 12.8% (n = 6) developed a plan between 3 and 5 years ago, and 14.9% (n = 7) of respondents reported developing a strategic plan more than 5 years ago.
Performance management and quality improvement
In 2010, 67% of respondents reported having a formal PM process in place compared with 76% in 2007. The depth of PM implementation was assessed. Respondents noted a decrease in both partially and fully implemented department-wide PM from 2007 to 2010. However, since 2007, the number of respondents reporting PM fully implemented in specific programs (rather than department-wide) increased by 3 respondents and did not change for PM partially implemented in specific programs.
The percentage of respondents who reported having a QI process in place also decreased slightly in 2010. In 2007, 82.4% of respondents reported having a QI process in place whereas 76.5% of respondents reported a QI process in place in 2010. Similar to PM, the depth of QI implementation was assessed. QI implemented department-wide and fully implemented for specific programs decreased in 2010. Three additional respondents (n = 20, 39.2%) reported a QI process partially implemented for specific programs in 2010 compared with 2007 (n = 17, 33.3%). Table 1 presents the level of PM and QI reported at state and territorial public health agencies.
In 2010, respondents were asked the number of formal projects the state/territorial health agency implemented to improve the quality of a service, process, or outcome in the past 12 months. A “project” was defined in the survey as “a systematic QI initiative that includes an aim statement; a work plan with tasks, responsibilities and timelines; intervention strategy(ies); and measures for tracking change” (B. Joly et al, unpublished data, February 2009). Thirty-three percent (n = 17) of respondents reported 1 to 3 projects, 29.4% (n = 15) of respondents reported 4 to 6 projects, 11.8% (n = 6) of respondents reported 7 to 10 projects, and 9.8% (n = 5) of respondents reported more than 10 projects in the past year. Approximately, 16% (n = 8) of respondents reported no implemented QI projects in the past year.
Respondents were asked to select all the frameworks or approaches to QI that their agencies have used in the past year. Fifty-two percent (n = 26) of respondents used the Plan-Do-Check-Act (PDCA) or Plan-Do-Study-Act (PDSA), 32% (n = 16) used no specific framework or approach, 28% (n = 14) used Lean, 24% (n = 12) used the balanced scorecard, 8% (n = 4) used Baldrige Performance Excellence Criteria (or state version), 8% (n = 4) used 6-Sigma, and 18% (n = 9) used other specific frameworks or approaches (including business process reengineering, NIATx, Model for Improvement, and the Step-Up Performance Management System).
Respondents performed a variety of activities in their efforts toward QI in the past year. Multiple activities could be selected. Examples of such efforts toward QI included 86.3% (n = 44) of respondents set measurable objectives, 80.4% (n = 41) obtained baseline data, 64.7% (n = 33) mapped a process, 60.8% (n = 31) identified root cause(s) of a problem, 49% (n = 25) tested the effects of an intervention, 47.1% (n = 24) analyzed the results of the test, and 11.8% (n = 6) reported none of the above activities were implemented.
A majority of state and territorial health agencies reported planning to seek accreditation under a voluntary national accreditation program. They were asked to indicate their level of agreement with the statement “Our state/territorial health agencies would seek accreditation under a voluntary national accreditation program.” Thirty-three percent of respondents (n = 16) reported that they strongly agreed, 39.6% (n = 19) of respondents agreed, 20.8% (n = 10) of respondents were neutral, 4.2% (n = 2) of respondents disagreed, and 2.1% (n = 1) of respondents strongly disagreed. As expected, the level of agreement with the statement “Our state/territorial health agencies would seek accreditation under a voluntary national accreditation program within the first 2 years of the program (2011–2012)” was slightly lower: 19% of respondents (n = 9) strongly agreed, 17.0% (n = 8) of respondents agreed, 42.6% (n = 20) of respondents were neutral, 17.0% (n = 8) of respondents disagreed, and 4.3% (n = 2) of respondents strongly disagreed.
Respondents reported discussing accreditation with the following entities: state/territorial health agency staff (86.3%, n = 44), local health department staff in the state or territory (80.4%, n = 41), staff in other state/territorial health agencies (52.9%, n = 27), other organizations/groups including Public Health Institutes, Public Health Associations, Schools of Public Health, and Improvement Councils/Taskforces (43.1%, n = 22), elected officials other than State Board of Health members (29.4%, n = 15), and State Board of Health (17.6%, n = 9). Five respondents (9.8%) reported that they had not discussed accreditation with any of the above entities.
The prerequisites for the application of health department accreditation under the Public Health Accreditation Board (PHAB) include the completion of a health assessment, a health improvement plan, and a strategic plan within the past 5 years of application.15 Among respondents who agreed or strongly agreed that they would seek accreditation (n = 35), approximately 63% (n = 22) of respondents have completed a state/territorial health assessment, 51% (n = 18) of respondents have completed a health improvement plan, and 74% (n = 26) of respondents have completed an agency-wide strategic plan. The 3 aforementioned prerequisites were each completed less than 5 years from when the survey was administered. Forty-two percent (n = 15) of the 35 respondents who agreed or strongly agreed that they would seek accreditation completed all 3 prerequisites within that 5-year timeframe.
For those who agreed or strongly agreed that they would seek accreditation in the first 2 years of the program (n = 17), 59% (n = 10) of agencies have developed a state/territorial health assessment, 41% (n = 7) of respondents have developed a health improvement plan, and 82% (n = 14) of respondents have developed a strategic plan. All 3 items were completed less than 5 years from when the survey was administered. Forty-one percent (n = 7) of the 17 respondents who agreed or strongly agreed that they would seek accreditation in the first 2 years of the program completed all 3 prerequisites.
Capacity and leadership
Approximately 55% of respondents (n = 29) reported having staff with dedicated time as part of their job description to monitor performance and QI work throughout the agency. A majority of respondents reported having 25% or less of their staff with formal training in QI methods. Seventy-two percent (n = 36) of respondents have between 1% and 25% trained, 12% (n = 6) of respondents have between 26% and 50% trained, no respondents reported between 51% and 75% trained, and 4% (n = 2) of respondents reported between 76% and 100% of staff members trained in QI methods. Twelve percent (n = 6) of respondents reported no staff members formally trained in QI methods.
Agencies support or encourage staff involvement in QI by engaging in the following activities: 62.7% of respondents (n = 32) stated that QI is included in job descriptions for some employees, 56.9% of respondents (n = 29) provide training to staff in QI methods, 35.3% of respondents (n = 18) recognize outstanding QI work with employee recognition award(s), 29.4% of respondents (n = 15) have QI efforts included as part of employee performance goals, 25.5% of respondents (n = 13) provide funding to support QI efforts, 23.5% of respondents (n = 12) formed a QI committee that coordinates QI efforts, 11.8% of respondents (n = 6) do not actively encourage staff involvement in QI efforts, 11.8% (n = 6) noted other activities such as participating in an informal QI network, and establishing a performance improvement manager position via the NPHII, and 3.9% of respondents (n = 2) provide monetary incentives to support or encourage staff involvement in QI efforts.
Leadership-educational background of the state health official
Sixty-three percent of state health officials (SHOs; n = 32) have an MD degree. Having a medical degree held by the SHO was unrelated to the practice of QI. Seventy-five percent (n = 24) of the 32 SHOs with an MD degree had a QI process in place and 75% (n = 15) of the 20 SHOs without an MD degree reported a QI process in place. Eighty-four percent (n = 27) of the 32 SHOs with an MD had 1 or more formal QI projects implemented in the past year compared with 80% of SHOs without an MD. Fifty-six percent (n = 18) of the 32 SHOs with an MD degree had staff with dedicated time in their job description to monitor performance and QI and 55% (n = 11) of the SHOs without an MD degree had such staff. Finally, 52% (n = 16) of the SHOs with an MD degree used a PDCA or PDSA cycle in the past year, compared to the 50% (n = 10) of the 20 SHOs without an MD that utilized a PDCA or PDSA cycle in the past year.
Despite limited reported variation in the practice of QI by SHO MD degree status, the agencies with SHOs with a medical degree did report a greater percentage of agreement that they would seek accreditation than those SHOs without the degree. Eighty-three percent (n = 25) of the 30 agencies with SHOs with an MD degree agreed or strongly agreed that they would seek accreditation compared with 55% (n = 10) of the 18 agencies with SHOs without an MD who agreed or strongly agreed they would seek accreditation. Thirty-eight percent (n = 11) of 29 agencies with SHOs with an MD agreed or strongly agreed they would seek accreditation in the first 2 years of the program compared with 33% (n = 6) of the 18 agencies with SHOs without an MD.
Forty percent of SHOs have an MPH degree (n = 21) and 60% of SHOs (n = 31) do not have the degree. All 21 SHOs with MPH degrees and 30 of the 31 SHOs without MPH degrees responded to the following questions. Seventy-one percent (n = 15) of agencies with SHOs with an MPH reported a QI process in place compared with 80% of agencies with SHOs without an MPH (n = 24). Forty-three percent of agencies with SHOs with MPH (n = 9) and 40% of agencies with SHOs without an MPH (n = 12) reported implementing between 4 and 10 formal QI projects in the past year. There is also minimal distinction in the reporting of a PM system by SHOs with an MPH (n = 14, 67%) compared with SHOs without an MPH (n = 20, 67%). Twenty-four percent of agencies with SHOs with an MPH (n = 5) reported participating in QI efforts as part of employee performance goals compared with 33% of agencies with SHOs without an MPH (n = 10); 43% (n = 9) with an MPH reported QI included in job description for some employees compared with 77% (n = 23) without an MPH; 10% (n = 2) with an MPH have formed a QI committee that coordinates QI efforts compared with 33% (n = 10) without an MPH; 24% (n = 5) with an MPH do not use any of the listed methods to actively encourage staff involvement in QI efforts compared with 3% (n = 1) of those without an MPH (Table 2). Also as noted earlier, PDCA/PDSA was the most frequently used QI framework in a state/territorial health agency in the past year: 38% (n = 8) of the 21 agencies with SHOs with an MPH used this framework compared with 62% (n = 18) of 29 agencies who responded that they have SHOs without an MPH degree.
State Health Officials with an MPH did not differ from SHOs without an MPH in terms of their level of agreement with the statement about intending to seek accreditation. Seventy-five percent (n = 21) of the 28 agencies with SHOs without an MPH agreed or strongly agreed to seek accreditation compared with 70% (n = 14) of the 20 agencies who responded with SHOs without an MPH.
The 2010 ASTHO Profile informs our understanding of state and territorial health agencies' participation in planning, performance management, QI, and interest in accreditation. Almost half of respondents developed a state/territorial health assessment within the last 3 years. More than half of respondents developed the strategic plan in the last 3 years. Since 2007, a greater percentage of respondents reported developing a SHIP in the last 3 years, thus aligning with the Healthy People 2020 Public Health Infrastructure Goal 15 to increase the implementation of health improvement plans.17
Department-wide implementation of a formal PM program decreased in 2010 from 2007 regardless of whether the PM program was fully or partially implemented (Table 1). Implementation of PM for specific programs slightly increased for fully implemented programs and did not change for programs that were partially implemented. Potential reasons for the department-wide decrease may include an improved understanding of PM terminology since 2007 and thus possibly more accurate reporting in 2010. It may also reflect a shortage of staff due to the recession to conduct the PM work. Recent Centers for Disease Control and Prevention NPHII funding has entirely changed the landscape on the use of performance management in states. Forty-nine of the 50 states are now funded to hire a performance improvement manager and implement a performance management program, including activities associated with accreditation readiness.
Respondents also reported a decrease in a QI process in 2010 compared with 2007 for most levels of QI implementation except for QI partially implemented for specific programs (Table 1). Possible reasons for the reported decrease in a QI process may include an improved understanding of QI and thus more accurate reporting, use of a slightly revised definition of QI in 2010 compared with 2007 and/or limited staff to perform QI due to the recession. When comparing the same survey question in 2007 to 2010, a self-reported QI process in place decreased from 82.4% to 76.5%. In 2010, however, the number of formal QI projects was assessed and based on response to this question, 84.3% of respondents had implemented at least 1 formal QI project in the past 12 months indicating a presence (albeit limited) of a QI process at the agency. The survey question regarding the number of formal QI projects implemented at the agency was among several new QI-related questions asked on the 2010 survey. These additional survey questions allow for greater exploration of the presence of organizational QI and alternate ways to assess the construct. The results indicate that the measurement of QI at public health departments needs continued enhancement and refinement on the profile surveys. Despite relatively high levels of self-reported QI or PM activities by health departments, it remains difficult to identify widespread QI practices or measurable outcomes related to those activities. Moreover, this bolsters the prior suggestion that our reporting may overestimate current efforts.24 Whether a reported QI process by respondents is 76.5% or 84.3% (ie, based on the implementation of at least 1 QI project), the result (similar to the 2007 findings) continues to lie somewhere between the 61.7% reported by Beitsch et al25 and 88% reported by Mays et al.26
The Healthy People 2020 Public Health Infrastructure Goal 16 is to increase the proportion of tribal, state, and local public health agencies that have implemented an agency-wide QI process.17 On the basis of the 2010 survey results, 11 respondents noted they have QI partially or fully implemented department-wide. Agencies are beginning to take action to encourage broader implementation of QI. Eighty-eight percent have at least some staff who received formal training in QI methods (72% have between 1% and 25% of their staff trained). Sixty-three percent of respondents have QI included in job descriptions for some employees and 57% of respondents provided training to staff in QI methods. With the implementation of NPHII, staffing and training at health departments in the areas of performance management will increase further.16
Seventy-three percent of respondents agreed or strongly agreed that they would seek accreditation and 36% agreed or strongly agreed that they would seek accreditation in the first 2 years of the program. Respondent interest in seeking accreditation may indicate the perceived value of accreditation to the agency. A majority of respondents have discussed accreditation with their own staff and with staff in local agencies in their state. Fewer respondents have discussed accreditation with elected officials or their respective board of health. The primarily internal discussions of accreditation may seem appropriate given its early developmental stage and parallel how innovation or new approaches are typically diffused. Innovators or early adopters of change (ie, accreditation in this case) tend to be moved by intrinsic motivation, so internal organizational discussion would make sense at this developmental stage.27 Discussion with external organizations, however, will need to happen to get appropriate buy in for accreditation and to be able to meet particular PHAB governance standards. As an example, one measure under PHAB Domain 9 “Evaluate and continuously improve processes, programs and interventions” is to “engage the governing entity in establishing agency policy direction regarding a performance management system.”8
In terms of accreditation prerequisites, a strategic plan was most frequently developed, followed by a state/territorial health assessment and health improvement plan, respectively. Two of the 3 prerequisites for accreditation (ie, use of SHIP and a strategic plan) were assessed in 2007 and 2010. Reported completion of the SHIP in the last 3 years increased between 2007 and 2010. Three additional respondents reported not developing a SHIP in 2010 compared with 2007. Two additional respondents reported completing a strategic plan (regardless of the timeframe) between 2007 and 2010. These results suggest that respondents are slowly preparing for accreditation and that an improved understanding of the prerequisites may be emerging. It is important to note that these prerequisites have not been evaluated against PHAB's definitions and self-report of their completion does not indicate that the documents may satisfy the PHAB prerequisite requirement. As health departments become accredited, however, reports of the ASTHO Profile data will also include health department accreditation status. This type of reporting is similar to the American Hospital Association's (AHA's) Guide to the nation's hospitals and health systems that documents both the self report of hospital and health systems performance from the annual American Hospital Association's surveys as well as the Joint Commission accreditation status.28 ASTHO is currently providing technical assistance to states related to the prerequisites and general accreditation planning.
Organizational leadership is necessary for performance improvement, implementation of QI, and interest in application for accreditation. Agencies with SHOs with an MD degree did report a greater percentage of agreement that they would seek accreditation than those agencies with SHOs without a medical degree. This finding may potentially be related to SHOs with a medical degree having greater experience with accreditation and certification within the larger health care industry.
Similar to the 2007 findings, limited variation was found in the practice of PM and QI by SHO MPH degree status in 2010.21 Additional encouragement and support of staff involvement in QI by all SHOs may be needed to foster a QI culture at health agencies. Few agencies have formed a QI committee or recognized staff for outstanding QI work. The establishment of performance improvement managers through NPHII will strengthen the capacity for this performance and QI role at state/territorial health agencies. However, without agency leadership support setting the tone for QI culture, the expectation that agency-wide QI will increase may be unrealistic. Possible exposure or training in the concepts of performance and QI and accreditation for the entire leadership of the state and territorial health agency may be worth exploring. ASTHO is funded by the Centers for Disease Control and Prevention to provide direct technical assistance for accreditation readiness including QI where appropriate. Over the last 2 years, ASTHO has provided technical assistance to senior staff in several state health agencies. Furthermore, through the Centers for Disease Control and Prevention NPHII funding, 5 public health partner organizations (ASTHO, National Association of County and City Health Officials, Public Health Foundation [PHF], American Public Health Association, and National Network of Public Health Institutes [NNPHI]) are providing direct technical assistance on all aspects of performance management, QI, and systems development. Other current avenues for training include courses by the PHF,29 Institute for Healthcare Improvement,30 State Health Leadership Initiative,31 etc. The School of Public Health at the University of Minnesota will offer a Graduate Public Health Certificate in Performance Improvement beginning in Fall 2011 where students will learn basic QI concepts and skills as appropriate to their role in their organization. This certificate addresses concerns voiced by the National Board of Public Health Examiners, the Public Health Accreditation Board, and the Council on Education for Public Health to provide more educational opportunities in performance improvement to working public health professionals.32
There were several limitations to this study. No territories responded to the survey in 2007. Two territories responded in 2010. The inclusion of the 2 territories does not explain the decrease in PM or QI since 2007. Measurement of the QI construct needs continued refinement. There were several issues regarding the measurement of QI in this study. First, the definition of QI provided in the 2010 survey was slightly revised from the 2007 definition. Second, construct validity may begin to be assessed with the inclusion of additional QI related questions. Specifically, these additional QI related questions on the 2010 survey facilitate an understanding that the survey question regarding the presence of an agency QI process may be unclear or misinterpreted by respondents. For example, 6 respondents indicated they had no QI process at their agency but implemented more than 1 formal project to improve quality, 7 respondents indicated having QI in job descriptions but no QI process, and 5 respondents provided staff training in QI but reported no QI process. When one state was contacted to clarify their response, the respondent commented that the question itself inquires about the “agency” having its own QI process. Their initial response was “no” despite having QI partially implemented for specific programs. As referenced in the “Methods” section, the response options to this question included whether QI was fully or partially implemented either department-wide or for specific programs. Perhaps rewording the question to ask the level of QI implementation may provide greater clarity. Finally, verification of actual improvements in quality and performance management did not occur. Validity and reliability of the constructs continue to be limitations.
The 2010 ASTHO Profile suggests that both the practice and research of PM and QI in state and territorial health agencies need continued development. From a practice standpoint, only 5 respondents reported 10 or more QI projects during the past year. Less than half of respondents tested the effects of an intervention and/or analyzed results of a test. Education and training on frameworks beyond PDCA/PDSA is needed. Targeting 100% of respondents to set measurable objectives, obtain baseline data, map a process, and identify root causes of a problem may be worthwhile practical goals for state health agencies. If these goals were to be achieved, then increasing activities that support or encourage staff involvement such as providing QI training, establishing employee recognition for QI work, incorporating QI participation as part of employee performance goals, or developing a QI committee may need to occur. National events currently focused on building the capacity of PM and QI in public health departments may help drive improvement in these practice areas.
Advancing applied research in QI at public health departments is also needed. In particular, strengthening the measurement of the QI construct is essential for meaningfully assessing current practice patterns and informing future programming and policy decisions. Grantees from the Public Health Practice–Based Research Networks and Building the Evidence Base for Accreditation and QI program contribute to strengthening the measurement of QI in public health.33,34 In addition, PHAB measures of QI plans and processes may also further inform our understanding of the construct and can serve as accountability measures for health departments, even as they provide more objective validation through accreditation site visits.35,36
Finally, advancements in practice and research cannot occur in isolation of one another. Continued dialogue between public health practitioners and researchers is critical to advance this field. Specifically, examining the interactions between practice-tested interventions and research-tested interventions that promote QI can strengthen the effectiveness of our public health programs and services and ultimately lead to improved health outcomes. Accreditation may serve as a fulcrum to strengthen both.