Share this article on:

Untangling Desirable and Undesirable Variation in Public Health Practice: Accreditation and Research Working Together

Mays, Glen P. PhD

Journal of Public Health Management and Practice: January/February 2014 - Volume 20 - Issue 1 - p 149–151
doi: 10.1097/PHH.0000000000000035
Accreditation Horizons: Commentary

This commentary discusses untangling desirable and undesirable variation in public health practice of the American public health system.

Department of Health Services and Systems Research, University of Kentucky College of Public Health, Lexington.

Correspondence: Glen P. Mays, PhD, Department of Health Services and Systems Research at the University of Kentucky College of Public Health, 111 Washington Ave, Lexington, KY 40536 (

This work was supported through the Public Health Practice-Based Research Networks Program and the National Coordinating Center for Public Health Services and Systems Research, funded by the Robert Wood Johnson Foundation (Grant No. 64676). Glen Mays also was supported through a Clinical and Translational Science Award from the National Institutes of Health (Award UL1TR000117).

The author declares no conflicts of interest.

For better and for worse, a defining characteristic of the American public health system has long been its heterogeneity. The mission, authority, resources, and capabilities of public health agencies vary widely from one community to the next. The division of labor between local, state, and even federal agencies, and between government and the private sector, shifts as one moves across geopolitical boundaries and from the city to the countryside. Some of this variation represents disfunction and inequality—a fact that led an influential Institute of Medicine committee 25 years ago to conclude that the American public health system was in disarray.1 At the same time, some of this variation represents public health at its best: the ability to tailor and adapt health protections to the specific needs, risks, values, and preferences of the populations served. Distinguishing desirable from undesirable variation in public health practice remains a central challenge for the field.

Accreditation is one of the newest instruments in the public health system's toolbox for reducing undesirable variation in practice, and perhaps also for promoting desirable variation. The ability to define standards of practice, measure conformity with standards, and create incentives for achieving conformity has proved to be a powerful force for quality improvement, accountability, and consumer protection in other areas of the US health system as well as in other governmental and private sectors.2 But success with accreditation requires developing the “right” standards and measures and combining them with the “right” incentives. Standardizing the “wrong” elements of practice and creating distortionary or inequitable incentives can reduce desirable flexibility and adaptation, create waste and inefficiency, and lead to poorer performance and outcomes.

So, how can the public health profession determine when and where to standardize versus customize? Learning from the existing variation across the US public health system is a powerful and extremely practical strategy, using both informal experiential learning and more formalized research. The field of Public Health Services and Systems Research (PHSSR) encompasses an expanding body of applied studies that seek to distinguish desirable from undesirable variation in public health practice and to test strategies for spreading the helpful and reducing the harmful variation. Research underway within this field is positioned to help the accreditation movement craft and adapt its standards and its incentives in ways that advance public health practice. A comprehensive accreditation research agenda now exists to guide this work,3 but 3 areas of inquiry merit special attention here.

Back to Top | Article Outline

Research to Inform Accreditation Standards and Measures

Studies that identify public health structures and processes that consistently lead to more effective and efficient agency operations can be used to develop new accreditation standards and to adapt and refine existing standards and measures. For example, a growing number of studies within the PHSSR field examine variation in organizational and governance structures and administrative practices among public health agencies, with the goal of identifying characteristics that help agencies implement evidence-based programs and policies more effectively and efficiently.4 Researchers at the St Louis Prevention Research Center recently conducted a systematic review of this literature and identified 11 structures and processes—collectively termed as administrative evidence-based practices-—that PHSSR studies have linked to improvements in the effectiveness and/or efficiency of agency operations.5 These administrative evidence-based practices include characteristics of agency leadership, workforce development processes, relationships maintained with other agencies and organizations, and financial practices. The Public Health Accreditation Board (PHAB) already includes standards and measures related to these characteristics, but the most consistent findings from these types of PHSSR studies can be used to refine existing standards based on whether the research points toward a need for greater specificity or greater flexibility.

As another example, an array of PHSSR studies examine variation in the implementation of community health assessment processes and community health improvement plans—2 elements of public health practice that have long been the subject of national professional recommendations and that are now included as prerequisites for accreditation under PHAB. Many of these studies are being conducted by public health practice–based research networks (PBRNs), which bring together state and local public health agencies, universities, and community partners to study innovations in practice.6 A recent study conducted by Wisconsin's PBRN reviewed the assessment and improvement plans completed in all 92 of that state's county public health jurisdictions.7 The study documented wide variation in practice but also identified some strategies that consistently resulted in more engaged partners and a higher likelihood of moving from planning to implementation. Similarly, a study underway by the Kansas Public Health PBRN is comparing the effectiveness and efficiency of 2 alternative approaches to conducting assessment and improvement planning processes: a traditional approach that involves independent processes in each county health department jurisdiction versus a regional approach that brings together multiple neighboring county jurisdictions. Findings from these types of studies can help PHAB refine its accreditation prerequisites and standards related to these practices, again by including greater specificity when this is justified by evidence, and greater flexibility when evidence is lacking or equivocal.*

Back to Top | Article Outline

Research to Inform Accreditation Incentives

Now that the implementation of a national voluntary accreditation program is underway in public health, a host of important questions emerge about what type of agencies pursue accreditation and under what circumstances, what incentives and disincentives for accreditation confront these agencies, and how do agencies respond to accreditation from possibilities that may include preparation, opposition, indifference or active avoidance. These issues have been explored carefully during the development of the national accreditation program to shape key elements of program design.8 , 9 In the postimplementation era, ongoing research can be used by a variety of public health stakeholders to adapt the implicit and explicit accreditation incentives that confront agencies in ways that discourage undesirable variation and promote desirable variation in practice. The PHSSR studies are examining many of these questions, such as work by the Nebraska Public Health PBRN that has documented wide variation among this state's largely rural public health settings in their efforts to prepare for accreditation and build capacity for quality improvement.10 Work underway by the Massachusetts PBRN is examining the effectiveness of regional public health service agreements in helping small public health jurisdictions prepare for accreditation, and exploring the extent to which accreditation and incentives provided through the Centers for Disease Control and Prevention's National Public Health Improvement Initiative lead agencies to join regional resource-sharing arrangements. Related studies by PBRNs in Colorado, Connecticut, Georgia, Nebraska, Ohio, and Wisconsin are examining related regional and multijurisdictional approaches to public health resource sharing and their utility in helping agencies prepare for accreditation.6 Findings from these types of studies are critically important for ensuring that accreditation incentives work to close rather than widen existing disparities in public health capacity across states and communities.

Back to Top | Article Outline

Research to Understand Accreditation's Impact

A question at the top of mind for nearly everyone within the public health profession concerns the impact of accreditation on public health practice and ultimately on population health. Here, we must insist on patience and caution in pursuing reliable evidence on this topic. Voluntary accreditation programs can be expected to generate powerful selection effects, particularly early in their implementation, by attracting some of the strongest candidates first. Simple comparisons of accredited and nonaccredited agencies are unlikely to yield reliable evidence about impact. Moreover, the national scope and high visibility of the PHAB accreditation program and its standards and measures are certain to create system-wide sentinel and spill-over effects on public health agencies across the nation, regardless of whether and when agencies decide to pursue accreditation. Clever and creative research designs and analytic strategies can be used to address some of these complications and derive insight about the changes in public health organization and practice that are attributable to accreditation, but these types of studies will be possible only after sufficient numbers of agencies have become exposed to the accreditation process over a sufficient length of time. In the meantime, qualitative case studies of agency experiences with and responses to accreditation are likely to generate early signals of practice changes. Moreover, studies of state-based public health accreditation programs such as in North Carolina and Missouri, which preceded the national PHAB program, offer some useful indications of impact on public health practice that likely have relevance for the national program.11

As for the question of accreditation's impact on population health, definitive research on this topic is likely many years away if it proves possible at all. The changes in organization and practice motivated by accreditation are likely to have broad yet diffuse effects on the adoption and implementation of many different individual public health programs and policies. These programs and policies, in turn, are likely to have broad yet diffuse effects on a variety of health-related behaviors, individual and community risk factors, and health outcomes over very different time horizons. Detecting diffuse effects on multiple risk factors and outcomes that are mediated through multiple and variable programs and policies over different time periods is a daunting task for the research community, and one that likely will become feasible only with multiple years of high-quality data pre- and postaccreditation.

In the nearer term, it will be much more important for the research community to help the accreditation movement learn (1) whether and how accreditation leads to changes in public health agency organization and practice and (2) whether and how these changes support the delivery of programs and policies that have been shown to protect health and prevent disease and injury on a population-wide basis. Evidence on these topics will enable continual improvements in the design and implementation of accreditation, which in turn will drive continuous improvements in public health practice.

Back to Top | Article Outline


1. Institute of Medicine, National Academy of Sciences. The Future of Public Health. Washington, DC: National Academies Press; 1988.
2. Mays GP. Can Accreditation Work in Public Health? Lessons From Other Service Industries [Working paper prepared for the Robert Wood Johnson Foundation]. Little Rock, AR: University of Arkansas for Medical Sciences; 2004.
3. Riley WJ, Lownik EM, Scutchfield FD, Mays GP, Corso LC, Beitsch LM. Public health department accreditation: setting the research agenda. Am J Prev Med. 2012;42(3):263–271.
4. Hyde JK, Shortell SM. The structure and organization of local and state public health agencies in the U.S.: a systematic review. Am J Prev Med. 2012;42(5)(suppl 1):S29–S41.
5. Allen P, Brownson RC, Duggan K, Stamatakis KA, Erwin PC. The makings of an evidence-based local health department: identifying administrative and management practices. Front Public Health Serv Syst Res. 2012;1(2): Article 2.
6. Mays GP, Hogg RA. Expanding delivery system research in public health settings: lessons from practice-based research networks. J Public Health Manag Pract. 2012;18(6):485–498.
7. Willems Van Dijk J. The quality of Wisconsin's community health assessments and community health improvement plans. In: Public Health PBRN Research-in-Progress Webinar. Lexington, KY: University of Kentucky; 2013. Accessed September 13, 2013.
8. Davis M, Cannon M, Corso L, Lenaway D, Baker E. Incentives to encourage participation in the national public health accreditation model: a systematic investigation. Am J Public Health. 2009;99(9):1705–1711.
9. Thielen A, Leff MG, Corso L, Monteiro E, Fisher J, Pearsol J. A study of incentives to support and promote public health accreditation. J Public Health Manag Pract. 2014;20(1): 98–103.
10. Chen LW, Nguyen A, Jacobson JJ, Ojha D, Palm D. Effectiveness and challenges for implementing quality improvement activities in Nebraska's local health departments. Front Public Health Serv Syst Res. 2012;1(3): Article 7.
11. Davis MV, Cannon MM, Stone DO, Wood BW, Reed J, Baker EL. Informing the national public health accreditation movement: lessons from North Carolina's accredited local health departments. Am J Public Health. 2011;101(9):1543–1548.

* More information on Public Health PBRNs and their research findings can be found at
Cited Here...

Copyright © 2014 Wolters Kluwer Health, Inc. All rights reserved.