Monteiro, Erinn MPH; Fisher, Jessica Solomon MCP; Daub, Teresa MPH; Zamperetti, Michelle Chuk MPH
In fall 2011, when the Public Health Accreditation Board (PHAB)1 launched national, voluntary public health agency accreditation for state, tribal, local, and territorial health departments (HDs), HDs were at varied readiness levels with regard to their ability to meet accreditation requirements. While some HDs reported they were prepared to apply as soon as the program began, others were still learning about the requirements and process.2 The majority of the nearly 2800 local health departments (LHDs) faced significant job losses and/or program cuts3 likely to impede their progress. Health departments also lacked an established system of financial support to undertake accreditation preparation activities. The National Public Health Improvement Initiative (NPHII), a cooperative agreement program that began in 2010, provided a new opportunity for HDs to make organizational and performance improvements such as those related to accreditation preparation; however, only 9 LHDs and 8 tribal entities met the eligibility requirements for this direct funding.4
To promote accreditation readiness among a greater number of HDs, the Centers for Disease Control and Prevention's (CDC's) Office for State, Tribal, Local and Territorial Support allocated specific funding, administered through the National Association of County & City Health Officials (NACCHO), to support HDs undertaking accreditation readiness* and quality improvement (QI) activities. Both the CDC and NACCHO, with input from partners including the Association of State and Territorial Health Officials, the National Network of Public Health Institutes, and the Robert Wood Johnson Foundation, collaborated to design a funding program titled the Accreditation Support Initiative (ASI).5
Two separate initiatives were developed, both designed to allow for maximum flexibility in the use of funds so as to address the unique needs of HDs. One initiative (the “big cities ASI”) was aimed specifically at large, metropolitan LHDs not funded through NPHII. Selected sites were funded both to complete readiness activities and to mentor at least one smaller LHD in their state or jurisdiction (“connector site”). A second initiative (“general ASI”) was aimed at any public health department† seeking to advance its own accreditation readiness. The general ASI was also aimed at “support organizations” (eg, state associations of county and city health officials, state health agencies, public health institutes) looking to provide accreditation-related technical assistance (TA) to constituent HDs.
Requests for applications were distributed in fall 2011 by invitation to 12 large, metropolitan LHDs for the big cities ASI and widely for the general ASI; 4 and 139 applications, respectively, were received. Applicants proposed their own scopes of work under 1 or more of the 4 broad categories of eligibility (Table 1). Both CDC and NACCHO reviewed applications for the big cities ASI; partner organizations assisted in review of general ASI applications. In all cases, applications were assessed for completeness; potential for accreditation readiness gains; development of transferable knowledge or practices; sustainability; and the appropriateness of budgets, work plans, and timelines.
Nearly $700 000 was awarded in funding to 18 sites (Tables 2 and 3). NACCHO administered the award funds and provided sites with resources and TA upon request and as capacity allowed. Both the CDC and NACCHO provided access to national accreditation and performance improvement networks and activities. All project activities were intended to stimulate QI and progress toward seeking national accreditation through PHAB. This article discusses the findings gleaned from review of the deliverables and data collected from the sites during year 1 of the ASI.
Several mechanisms were used to collect information from ASI sites during the project period to measure their progress in preparing for accreditation. First, each site (n = 18) had a contractual scope of work that included deliverables tailored to their specific activities. Second, sites submitted tools, example PHAB documentation, and other resources developed for posting on the NACCHO Web site as a means to share them broadly. Sites had the option to have deliverables de-identified before posting online. Third, funded local and tribal sites (n = 13) were asked about their expected PHAB application date during both the ASI application process (September 2011) and again at the end of the project (May 2012), allowing for a comparison in accreditation application timeframe over this 8-month period.
Fourth, the field has much interest in determining the amount of resources (time, workforce, and costs) needed to pursue accreditation.6,7 For this reason, NACCHO developed a time-cost questionnaire to collect information from the local and tribal sites on estimated resources spent on accreditation preparation, including related work done outside of the scope of this particular funding. The questionnaire also asked sites to report whether they had a designated accreditation coordinator and the percentage of time that person spent on accreditation or QI activities. Understanding that context and processes vary by agency, the questionnaire was not intended to inform best practices or recommendations on what agencies ought to contribute toward readiness activities but rather to identify any trends. Sites completed the questionnaire at 3 intervals during the 6-month project period. Data were manually entered into a spreadsheet and aggregated by site.
Fifth, NACCHO surveyed all sites briefly at the initiative's conclusion to get anonymous feedback about their experiences as an ASI site. The survey included 5 questions related to sites' self-reported benefits to participating in the ASI. All questions were rated on a 5-point scale (strongly disagree, disagree, agree, strongly agree, or N/A), with the option to provide an open-ended comment. The survey was implemented through the Zoomerang software, with the link e-mailed to all sites. The survey was open for 2 weeks, and a reminder e-mail was sent 2 days before its closing.
Finally, all sites submitted final reports at the end of the initiative. NACCHO provided templates and instructions to ensure consistency in the type of information reported. Reports included 2 components. Part 1 was a scope of work chart summarizing the funded activities of the sites during the project period, deliverables produced as a result, and overall outcomes for their agency. Part 2 was a narrative story reflecting on the benefits, challenges, and key strategies/lessons learned from participating in the ASI. The big city sites also described the successes and challenges in working with their connector site(s). All ASI sites were made aware that part 2 of the report would be posted to naccho.org and otherwise shared publicly.
Site activities and deliverables
On the basis of a quantitative and qualitative analysis of part 1 of the sites' final reports, activities generally focused on planning, training, and developing documents, processes or systems required by the PHAB Standards and Measures, Version 1.0. The most common activities supported by the funding were trainings or work related to community health improvement or strategic planning (PHAB Domain 5) and QI/performance management (PHAB Domain 9). Several agencies completed general work plans for undertaking the accreditation process and documentation review; other agencies communicated with governing boards and staff and reviewed public health laws. Five LHD sites applied for accreditation during the project period, with 4 using ASI funding toward PHAB fees.
In addition to these tangible outcomes, a number of sites cited perceived increases in performance and QI activities, key aspects of PHAB Domain 9 (to evaluate and continuously improve processes, programs, and interventions).7 Related activities included implementing discrete QI projects and developing aspects of a performance management system. Support organizations and big city ASI sites developed templates or practices to use when providing TA and mentorship to other HDs.
Data from the questionnaires showed that the number of hours spent completing specific accreditation preparation tasks ranged from 1 to more than 1000 hours, with no discernible trend based on the full-time equivalent rate for an accreditation coordinator or similar position. Results were also analyzed on the basis of population size served, again with no apparent patterns. Information as to the value of these data is presented later in this article.
Expected PHAB application date
Six HDs indicated in their ASI applications that they intended to apply to PHAB before the end of the project period. Five did apply in that timeframe, whereas the sixth HD's PHAB application was delayed because of unfinished prerequisite documents. Of the 7 remaining HDs, by the end of the initiative, 2 still expected to apply within their originally specified timeframe, 4 anticipated further delays in their application date, and for 1 site the data were not provided. Anecdotal feedback suggests that when timelines were adjusted, it was due to having gained a clearer understanding of PHAB processes and requirements and thereby being able to establish more realistic activity timeframes.
Fifteen of the 18 funded sites completed the brief electronic survey, including all 5 of the support organizations and 10 of the 13 local/tribal sites. The majority of sites agreed that they perceived an increased ability to serve their jurisdiction, improved staff communications, and strengthened relationships with other organizations. Most notably, all sites “agreed” (20%) or “strongly agreed” (80%) that this funding had made an impact on the accreditation readiness of the HD (or the HDs they supported through the work) in a way that would not have occurred without the funding.
Within the survey's open-ended comments, many sites noted the impact of the funding on leveraging momentum; with funding tied to submission of deliverables, staff members were more engaged and agencies felt more committed to adhere to activity timeframes when faced with competing priorities. Several sites noted the impact of the funding in building staff capacity by supporting staff time, travel, and trainings. Finally, a number of sites referenced the peer learning benefits experienced through a conference call with other ASI sites, mentoring and sharing resources with others, and attending national conferences or webinars.
Narrative report reflections from local and tribal sites
Final narrative reports written by the 13 participating local and tribal sites revealed important insights on the benefits, challenges, and lessons learned from participating in a funded initiative such as this.
Overall, some of the immediate benefits that sites observed included accelerated efforts toward accreditation by dedicating staff time; enforcement of activity timelines and deadlines; funding for PHAB fees and other resources; staff trainings to set the groundwork for future work; and networking with other HDs to help promote the sharing of ideas and resources. Sites reported that participating in the ASI helped them better understand the costs associated with accreditation preparation to help budget for future expenses; develop a realistic roadmap toward PHAB application; and increase awareness of requirements and expectations among staff and community members.
Most challenges identified related to time. Many sites noted that the short duration of the project, compounded by administrative procedures (ie, contracting, invoicing) and initially underestimating timeframes, resulted in either modifications to the work plan or some aspects of their work feeling rushed. Time was even more limited for sites using project funds for consultants, due to additional subcontracting considerations. Finally, time was also cited in relation to staff capacity for the work. While the funding did allow sites to dedicate support for accreditation staff, it also exposed that without this support, it would be difficult to “add (accreditation) responsibilities to already busy workloads,” citing existing budgetary and workforce challenges.
To address such challenges, sites reported several key strategies and lessons learned, starting with the importance of leadership knowledge and support. To adhere to an accreditation work plan, especially related to a funding award, sites agreed that support from agency leadership is critical in gaining buy-in from both the governing entity and the fellow staff. This support is especially important when trying to make progress in shifting the culture of quality in an organization. If funds are used specifically for QI projects, sites recommended using QI champions in the organization to advance change.
Sites emphasized that all staff members ought to be involved to some degree in the accreditation process, especially when faced with strict timelines and deadlines. To engage staff, sites suggested it could be helpful to give assignments and provide regular updates to ensure these efforts are always present in their minds. One site noted that it could be particularly helpful if the accreditation coordinator is assisted by a senior staff member who has knowledge of agency functions and operations. Another noted that funding allocated to hire short-term administrative support could assist in freeing up staff time to participate in the process. Sites further noted that communication and education about the process and the work being done are imperative: keep leadership informed of progress; have a plan for strategically communicating with staff at all levels; and find creative ways/use multiple venues for doing so. Finally, many sites noted important strategies related to relying on existing resources and expertise: use proven models and frameworks whenever possible; solicit expert advice; and continue to cultivate relationships with peers to share templates, documents, and advice.
Given that starting the accreditation preparation process can be overwhelming, one site advised focusing first on structural changes within the organization that have the highest potential for sustainability, such as building a carefully thought-out performance management system, a required element of PHAB Domain 9.8 As this site noted, performance management can be “labor- and resource-intensive, but is a long-term investment” that may pay off in improved quality, reporting, and performance.
Big city ASI sites found it sometimes difficult to establish the initial connection and maintain the dialogue with connector sites, given competing demands and schedules; this was noted as especially difficult when working with more than 1 site. All big city ASI sites noted that, to some degree, the connector-site relationship ended up being more of a mutual exchange of information rather than simply a 1-way provision of TA. One site noted that the connector-site concept “seems to be based on the underlying assumption that larger LHDs are better-equipped to provide TA, but that may not necessarily be the case,” suggesting that all HDs, regardless of size or level of accreditation readiness, may have lessons and best practices to share with one another.
Narrative report reflections from support organization sites
The final narrative reports from the 5 support organizations raised additional considerations regarding the use of funds to assist HDs in preparing for accreditation.
Support organizations framed the reported benefits of their participation in the ASI in 2 general ways. One type of benefit is for the HDs, including ones with which the support organizations worked directly and those HDs that the organizations will work with in the future. As noted, through these support organizations, a number of HDs across the country were able to address PHAB documentation and process requirements. In addition, the templates, models, and frameworks established by the support organizations can be used by other HDs beyond this project. The second type of benefit is for the support organizations themselves, as they noted that this opportunity increased their own knowledge and provided ideas for future TA opportunities in this area.
As with the local and tribal sites, support organizations faced challenges with time and timing. They all found that difficulty in coordinating schedules between multiple HDs, especially for on-site TA or group meetings, could significantly impact project timelines. Using existing trainings and information can be a good start; however, because of vast differences in terms of HDs' stages of accreditation readiness and understanding of certain processes and terminology, nearly all support organizations found it necessary to individualize the TA and feedback. This can further compound the already challenging time constraints.
Support organizations expressed different perspectives in terms of selecting recipients for the TA. One organization that surveyed potential recipients to gauge interest noted that those further along in accreditation preparation might be more likely to respond. This may result in not reaching agencies most in need of TA. On the contrary, acknowledging the time barrier issue, another organization noted that it might be wise to select recipients that are the most enthusiastic about receiving support and improving their processes, regardless of level of need.
The main takeaway all of the support organizations cited was that every HD is different in terms of structure, amount of leadership support, levels of performance improvement knowledge, and TA needs. As one organization noted, for TA to be truly effective, it is important to “start where their [HD] needs begin and not where we [support organizations] feel it would do the most good or would best fit our own schedule.” While acknowledging the reality of time and scheduling barriers, all sites emphasized that taking the time to determine the best TA approach is critical, even if that means revising the project plan.
Discussion and Conclusions
The concept of the first iteration of the ASI was well received by the field. The number of applications received indicates significant interest among HDs in even relatively small, one-time funding awards to support their accreditation readiness efforts. The flexibility of ASI activities allowed the funded sites both to direct attention toward critical needs and to improve practices consistent with meeting PHAB requirements.
Given the small sample size of funded sites and the scope of this article, recommendations about future iterations of the ASI cannot be made. As noted, many sites indicated that some work would not have been possible without this funding. However, because funding was often used to augment or accelerate existing work, it is difficult to differentiate specific advancements that may still have been made if not for participation in this effort. Still, it is clear that some improvements in accreditation readiness were made regardless of starting point and that flexible support for addressing HDs' unique needs can be a helpful strategy in accreditation preparation.
Data obtained through the time-cost questionnaires were inconsistent and widely varied, raising concerns related to reliability and interpretation of the findings. Especially with such a small sample size, these concerns suggest that the time-cost issue requires a more in-depth study than is feasible through the ASI. However, it is clear that much work related to performance improvement and accreditation preparation in local and tribal HDs simply requires time: for staff training and education, for new concepts to be absorbed and put into practice, for changes in both the system and within people to occur, for QI projects, and more. It is important for this to be acknowledged and understood at all levels: HDs, support organizations, partner organizations, and funders. Similar funding opportunities should carefully consider what could reasonably be accomplished during a project's timeframe and given an applicant's capacity.
In mid-2012, the big city and general ASIs were renewed for a second year. The purpose and structure remained largely the same, with some improvements based on the evaluation results discussed in this article. A comprehensive mixed-methods evaluation of both years of the ASI, including a focus on outcomes, was underway at the time this article was written. With the increased number of sites and data points available for the study, it is expected that stronger conclusions on the impact of the ASI and recommendations for future funding will be available in late 2013.
© 2014 Lippincott Williams & Wilkins, Inc.