On October 18, 2018, Mary Dixon-Woods gave the Harveian Oration marking the 500th Anniversary of the Royal College of Physicians.1 Professor Dixon-Woods, Director of The Healthcare Improvement Studies Institute and Co-editor of BMJ Quality & Safety prodded “QI is pervaded by optimism bias ... the Lovely Baby syndrome.” We know clinical safety hazards are to be fixed, post-haste, not subjected to years-long randomized studies, nor “problems to be admired.”2
With an international perspective, Hirschhorn et al.3 highlighted the gap between “QI implementers” and “QI researchers” cataloguing challenges: spread, evidence of scientific rigor, and publication. They recommend both “embedded (practice-based) researchers” and “implementers engaged in research.” To wit, Dr. Helen Crisp reported in The Lancet,4 The Health Foundation (UK) supports a new 3-year QI Science Research Fellowship. Moreover, Portela et al.5 reviewed quality improvement (QI) methods from PDSA (plan-do-study-act) cycles and before-after comparisons, to robust approaches: practical clinical trials, stepped-wedge designs, and qualitative or mixed-method tools.
The above observers and authors are all extolling QI leaders to promote the robust evaluation of QI interventions, and to support the spread and publication of that rigorously evaluated QI work. These same QI leaders, however, may wonder exactly where and how to start building systems to support their frontline teams in the pursuit of QI publication.
EXAMPLE: A TALE OF 2 QI TEAMS
Team 1, a multidisciplinary group led by an early career internal medicine staff physician and a motivated internal medicine resident, applied established QI methods to reduce, in hospitalized patients, administration of ineffective medications. At the outset of their QI project, the team incorporated important elements of scholarship into their project work: authorship discussions were completed and clear expectations of writing the manuscript and targeting appropriate journals were established. They reviewed the Standards for Quality Improvement Reporting Excellence (SQUIRE) guidelines6 and carefully designed a plan to study the impact of their interventions. Their efforts resulted in sustained improvements in several elements of the triple aim, and the team's success, highlighted by their published manuscript, was celebrated throughout the hospital and local community. Their publication opened new career opportunities for involved team members, particularly the resident. Their team success, bolstered by spread and publication, represented better outcomes for patients, advancement of improvement science, and accelerated professional development for QI team members.
Team 2 was similarly composed of a talented, multidisciplinary team led by an early career internal medicine physician and trainee, working on the important problem of increasing rates of advanced directive completion in their outpatient clinic. This team, like team 1, applied established QI methods to their project work, and like team 1 their dedication to sound QI methodology resulted in improved performance. Team 2, however, did not seriously discuss publishing their work until the project work was nearly completed. When they did eventually review their options for preparing a manuscript, the team members realized that simple changes to the study of their interventions, such as adding a similar clinic not involved in the QI project, as a control group, would have not only made the project more likely to be published, but would have strengthened the inferences and learning they were able to draw from their project results. Not having thought much about the study of their interventions during the project, they presumed a simple before-after study would have been adequate; however, had they pursued publication from the outset of the project, they may have considered other methods or study designs to enhance causal inference. Finally, the team members, having nearly finished their QI project, were eager to move on and after a few meetings ultimately decided not to submit a manuscript. The opportunities for improved patient outcomes, advancing improvement science, and accelerated professional development enjoyed by team 1 as part of their pursuit of QI publication went partially unrealized for team 2.
Publication can be a strong source of motivation, a tangible marker of success, and is widely recognized as important to career advancement within academic institutions.7,8 Pursuit of publication also allows individuals to develop professional networks, both within and outside the organization, which can lead to fruitful future collaborations, rich learning, and career development opportunities. This motivation may add incentive for QI teams, particularly those involving trainees, to persevere during project setbacks, which might otherwise deflate their efforts. QI organizations invested in advancing a culture of continuous improvement should consider implementing organized and formal efforts to support QI scholarship among the teams they train and support. Providing structured support for QI teams pursuing publication can result in more team 1 outcomes with frontline staff completing higher impact QI work, vigorously advancing improvement science, and enjoying enhanced professional development.
DEFINING SCHOLARLY QI WORK
Boyer9 has described 5 separate dimensions of scholarship.10 This framework, used by academic promotion committees in comparing different categories of scholarly work, can also be helpful in codifying the value and distinction between scholarly QI work, categorized as the scholarship of application, and traditional biomedical and health services research, categorized as the scholarship of discovery. A fundamental distinction in Boyer's description of the scholarship of application, when compared with the scholarship of discovery, is that in the scholarship of application extant knowledge is translated into practice amid the complexity of differing social structures and human behavior. Considering whether publication efforts fall within the framework of the scholarship of application can help distinguish QI from other forms of traditional research and encourage QI as a uniquely worthy scholarly pursuit.
For work to be considered QI, it is most often motivated by clear knowledge of an important performance gap to be mitigated, is pursued using QI tools, which provide deep knowledge of the system to be improved, and addresses that performance gap through iteratively developed interventions, which are carefully studied. QI publication is when teams completing QI work go on to prepare a manuscript describing their efforts, sharing the results of those efforts while distilling learning for those in other clinical microsystems to learn from. Those teaching traditional concepts of QI in health care11–13 will be well served in deliberately defining scholarly QI work narrowly to reflect some of these important differences between the scholarship of discovery and the scholarship of application. When the rigid definition of scholarly QI work selected aligns with the broader vision for continuous improvement, it can provide greater support for the organization's desired approach toward QI. The Table provides examples of project work that falls into, or outside of, this construct of QI.
Hypothetical Classification of Scholarly Quality Improvement
and Traditional Biomedical Research
|Team applies model for improvement to reduce low-value testing in chest pain evaluations
||Begins with intention to iteratively close a local performance gap using an established QI methodology
|Team evaluates outcomes of a hospital readmission reduction program
||Health services research
||Does not follow an iterative improvement process
|Team performs a systematic review of different QI methodologies
||Does not begin with intention of closing a local care gap
|Team explores the experience of patients with limited English proficiency
||Qualitative Methods can be employed in preliminary work when searching for a deeper understanding of the local problem
|Team used DMAIC approach to improve vaccination rates
||Begins with an intention to iteratively close a local care gap using established QI methodology
Abbreviations: DMAIC, define, measure, analyze, improve, and control; N/A, not applicable; QI, quality improvement.
CHALLENGES IN ADVANCING PUBLICATION OF QI
Challenges to sustainably integrating QI into a modern health care organization's daily work have been well described,14–16 but organizations, QI educators, groups, or departments (hereafter QI organization) seeking to assist teams in QI publication will face additional challenges. In addition to limited familiarity with the nuances of QI writing17 and lack of experience with publication processes among busy, often clinically active QI practitioners, the QI organization will need to help teams address other common challenges to successful QI publication.
Inexplicit initial assessment of publication goals
Frontline QI teams often do not sufficiently plan for the dissemination of their work to a broader audience, including publication in a peer-reviewed journal, at the start of their project. Most project charters and other structured QI project tools focus on the fundamental QI work, and teams often start their project with only a vague desire to publish their work. Lack of transferability of the work to other settings or lack of novelty, limited team resources, or competing priorities may mean that pursuing publication is unwise or unrealistic even as the project work is still necessary and valued. Having a candid conversation on publication goals at the start of the project is important to better focus the team's efforts. For teams pursuing publication, related discussions around important issues necessary for successful publication including timelines, commitment to the project work, authorship, and publication venue are generally best conducted early in the project.18–20
Teams start writing too late
Regardless of the depth of discussion around a team's initial publication goals, significant challenges arise when teams do not begin preparing their manuscript until after they have completed their QI project. As in the case of team 2 at the beginning of this article, when a project is completed, team members will have variable recall of important details of their interventions, fluctuating levels of motivation, and other competing priorities to balance. After completion, a sense of the project “being done” may contribute to difficulty in persevering through the arduous process of preparing a manuscript. Under these circumstances, it is very easy for the writing process to become disorganized and fragmented. Perhaps most importantly, writing the manuscript after your interventions are completed may lead to less careful thought given to the study of those interventions compared with early pursuit of publication where authors will carefully consider the study design necessary for successful publication prior to beginning their interventions.
Difficulty finding time, data expertise, and mentorship
Inadequate time for performing both the regular work of the QI project and the additional work necessary for publication is often a team's most obvious and significant obstacle. Funding which may be available in the research space is often much more limited for those interested in completing and publishing QI work within a local microsystem. While intramural sources of funding for improvement work may be available, procuring these funds requires additional time and effort and is not guaranteed. Even when these funds or dedicated time to work on a project are available, it is likely that much of the time spent preparing a manuscript will still fall outside of normal working hours.
Finding expertise and support in data collection and analysis is another significant challenge. Successful publication may require larger volumes of data or specific data that are difficult to obtain without institutional electronic health record support. Often, teams are left to do the best they can due to limitations of electronic data availability for which the team may depend on manual data collection, or quality assurance data that may not be adequately rigorous for publication in quality and safety journals. Expertise in QI data collection and analysis including statistical process control may not be readily available. While colleagues with experience and expertise may be available and willing to collaborate and assist with data collection, analysis, and publication, finding the connection to these individuals may be daunting for the early-career or nonphysician QI project leader.
Navigating the institutional review board (IRB) and determining whether a review is required is often a confusing topic for QI teams21 and there may be varying levels of familiarity with QI among IRB staff.22,23 Even among experienced QI practitioners, finding journals that understand and publish QI may feel daunting or confusing. Elements of QI writing, which are necessarily different from research, including communication of subtle contextual elements, the application of formal and informal frameworks for understanding system performance, and QI study designs may not be familiar to those primarily trained in sound QI methodology or those coming from a background rooted in the scholarship of discovery.
WHAT THE HEALTH CARE ORGANIZATION CAN DO
In spite of these challenges, there is much that a QI organization can do to support motivated QI teams interested in publication. In the Figure, a driver diagram illustrates structured actions that can be taken to address some of the common challenges faced in advancing the publication of QI work.
Assist teams in an early and realistic assessment of scholarly ambitions
Finding ways to assist QI teams in planning for publication from the point of project origination is critically important. Initially, this may involve simply eliciting interest in publication from the team leader when a project charter is submitted to the QI organization. If a formal system of tracking new QI projects is not being used, periodically asking practice leaders to refer teams to the QI organization for an initial consultation may help identify those interested in publishing their QI work. QI leaders can start small, just touching a few teams in the early stages of developing a more formal, systematic process of outreach to teams starting a new QI project. Once the project team connects to the organization for support, an assessment of scholarly viability should be conducted. This initial assessment may involve honest questions about the following: the external importance of the work; the relevance of the targeted improvements to other organizations; the reliability and sustainability of data collection; departmental support and resources available; formal stakeholder analysis; and a team's competing priorities and motivation to put forth the discretionary effort needed to see the project through to publication. The QI organization or individual leading efforts to support institutional QI publication can maintain a catalog of potential funding opportunities and resources and can share some of these with the project team at this initial meeting; this catalog of resources may also prove helpful in establishing a longitudinal partnership between the QI organization and that frontline QI team.
The initial planning meeting is also a good time for the team to become familiar with the SQUIRE guidelines1 and to think specifically about venues for publication. Often, the introduction section of the manuscript along with the background research could be written and submitted with the project charter. Including this work, along with publication goals, will not only make the project charter a stronger document, it can also help gain key stakeholder support. Once the journal submission target is agreed upon, the team can plan to incorporate appropriate elements of SQUIRE into their proposed QI methodology and set timelines for their writing. Using primers on difficult tasks such as authorship discussions,18,19 conflict resolution,24 and running effective meetings25 could be considered as part of the team's initial project planning work.
Formalize scholarship as a component of the everyday project work
Teams should be encouraged to concurrently write their manuscript as part of their regular QI work. Continuing to write as the project progresses and using hypothetical tables for hypothetical data display and intervention descriptions, will help teams stay organized and make the work remaining at project completion feel less daunting. In the early phases of the project work, careful attention should be given to how the team will ultimately study their interventions, thinking closely about study design, measurement approach, and addressing potential confounding variables. Writing milestones can be added to traditional project management tools. For the everyday QI project work, setting formal meetings with concise minutes and agenda items, specific and clear action items including those related to scholarship and supported by mutually agreed-upon check-ins, and deliverables between the project team and the organizational support staff will help promote accountability in the writing process.
In the academic medical center environment, learners often have expectations or requirements both to publish academic work and to meaningfully participate in QI work. Encouraging project teams to include trainees, who productively utilize these academic motivators, will help the team better integrate work advancing publication into the main project work, particularly as trainees may not have the deep knowledge of the system or stakeholders to be intimately involved in some of the other QI work. For those working outside of the academic medical center, aligning scholarly work with high-profile efforts around quality or patient safety or addressing issues important to the community will help gain support for that specific effort and for future QI work. Including hospital leaders in the publication efforts may also help gain motivation and support for the project work and facilitate communication of your work to the hospital and local communities. An initial publication success can be particularly valuable in opening the door to securing future resources and support from institutional leadership for other QI efforts.
Organizing and building resources around QI publication
The lack of time many teams face may be a major opportunity for the QI organization to build support for its approach to advancing QI scholarship. Working with practice leadership to develop small grants programs or time-based awards, which avail QI authors time to work on their manuscript, may provide an opportunity for the busy but motivated frontline QI team to publish. Designing the awards so that they follow a framework of scholarship highlighting the important concepts of beginning early and intentionally and integrating writing into the standard QI workflow will help set teams up for success with both their current project and future QI efforts. Providing other targeted resources, where possible, like data collection and analytics, support in QI study design, and manuscript editing, can also further engagement between frontline QI teams and the QI organization. In organizations where a small team can be assembled to assist with study design, data collection, and analytics, the results may be compounded across many departments across the organization yielding a robust return on investment.26
Establishing an official venue or group for those interested in publishing QI work can help to develop both a content resource and a database of possible mentors and collaborators for those interested in gaining expertise in writing about their QI work. After resources are in place and teams self-select around interest in QI scholarship and award applications, a natural community can begin to develop around QI scholarship. Official sanctioning and support of this venue by the organization will send the message that this work is both important and valued. The venue could be a journal club, grand rounds, interest group, or quality forum. Simple incentives like providing food or blocked clinical calendar time to attend may be helpful in gaining early engagement. This type of forum may also help promote other efforts within the larger QI organization such as increasing participation in existing quality certification and training programs. Organizations seeking to establish an infrastructure to support publication of QI work can look to the evolution of other internal clinical, education, and research-related groups to learn local best practices in developing academic communities. Once the community is established, those seeking mentorship or guidance will have an obvious venue where they can seek input or collaboration from others sharing this interest in QI publication.
While health care organizations benefit tremendously when frontline staff apply a scholarly approach to QI work, specific obstacles exist for teams interested in pursuing publication of their QI work. Health care organizations can support frontline staff by helping to recognize, guide and plan scholarly ambitions early in the project work, formalizing scholarship as a part of the QI project work and by building and connecting the QI team to necessary resources, particularly by supporting authors', time, and connecting them to a community oriented around QI scholarship. Engaging staff at the point of project conception is both difficult and critical to those efforts. Additional efforts to correlate these recommendations with scholarly outcomes will be important in validating these recommendations.
1. Dixon-Woods M. How to improve healthcare improvement—an essay by Mary Dixon-Woods. BMJ. 2019;367:l5514.
2. Sunstein CR. The real world of cost-benefit analysis: thirty-six questions (and almost as many answers). Columbia Law Rev. 2014:167–211.
3. Hirschhorn LR, Ramaswamy R, Devnani M, Wandersman A, Simpson LA, Garcia-Elorrio E. Research versus practice in quality improvement
? Understanding how we can bridge the gap. Int J Qual Health Care. 2018;30(suppl 1):24–28.
4. Crisp H. Building the field of improvement science. Lancet North Am Ed. 2015;385:S4–S5.
5. Portela MC, Pronovost PJ, Woodcock T, Carter P, Dixon-Woods M. Republished: how to study improvement interventions: a brief overview of possible study types. Postgrad Med J. 2015;91(1076):343–354.
6. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for Quality Improvement
Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986–992.
7. Atasoylu AA, Wright SM, Beasley BW, et al. Promotion criteria for clinician-educators. J Gen Intern Med. 2003;18(9):711–716.
8. Thomas PA, Diener-West M, Canto MI, Martin DR, Post WS, Streiff MB. Results of an academic promotion and career path survey of faculty at the Johns Hopkins University School of Medicine. Acad Med. 2004;79(3):258–264.
9. Boyer EL. The scholarship of engagement. Bull Am Acad Arts Sci. 1996;49(7):18–33.
10. Seifer SD, Wong K, Gelmon SB, Lederer M. The community-engaged scholarship for health collaborative: a national change initiative focused on faculty roles and rewards. Metrop Univ Jl. 2009;20(2):5–21.
11. Varkey P, Reller MK, Resar RK. Basics of quality improvement
in health care. Mayo Clin Proc. 2007;82(6):735–739.
12. Shojania KG, Grimshaw JM. Evidence-based quality improvement
: the state of the science. Health Aff. 2005;24(1):138–150.
13. Jones B, Vaux E, Olsson-Brown A. How to get started in quality improvement
. BMJ. 2019;364:k5408.
14. Abdallah A. Implementing quality initiatives in healthcare organizations: drivers and challenges. Int J Health Care Qual Assur. 2014;27(3):166–181.
15. Dixon-Woods M, McNicol S, Martin G. Ten challenges in improving quality in healthcare: lessons from the Health Foundation's programme evaluations and relevant literature. BMJ Qual Saf. 2012;21(10):876–884.
16. Marshall M, De Silva D, Cruickshank L, Shand J, Wei L, Anderson J. What we know about designing an effective improvement intervention (but too often fail to put into practice). BMJ Qual Saf. 2017;26(7):578–582.
17. Stevens DP. Writing to Improve Healthcare: An Authors Guide to Publication. Boca Raton, FL: CRC Press; 2018.
18. Huth EJ. Guidelines on authorship of medical papers. Ann Intern Med. 1986;104(2):269–274.
19. Dance A. Authorship: who's on first? Nature. 2012;489(7417):591–593.
20. Wardale D, Hendrickson T, Jefferson T, Klass D, Lord L, Marinelli M. Creating an oasis: some insights into the practice and theory of a successful academic writing group. High Educ Res Dev. 2015;34(6):1297–1310.
21. Gautham KS, Pearlman S. Do quality improvement
projects require IRB approval? J Perinatol. 2021;41(6).
22. Lee SS-J, Kelley M, Cho MK, et al. Adrift in the gray zone: IRB perspectives on research in the learning health system. AJOB Empir Bioeth. 2016;7(2):125–134.
23. Lindenauer PK, Benjamin EM, Naglieri-Prescod D, Fitzgerald J, Pekow P. The role of the institutional review board in quality improvement
: a survey of quality officers, institutional review board chairs, and journal editors. Am J Med. 2002;113(7):575–579.
24. Deutsch M, Coleman PT, Marcus EC. The Handbook of Conflict Resolution: Theory and Practice. San Francisco, CA: John Wiley & Sons; 2011.
25. O'Dea N, de Chazal P, Saltman D, Kidd M. Running effective meetings: a primer for doctors. Postgrad Med J. 2006;82(969):454–461.
26. Horwitz LI, Kuznetsova M, Jones SA. Creating a learning health system through rapid-cycle, randomized testing. N Engl J Med. 2019;381(12):1175–1179.