Secondary Logo

Journal Logo

Foreword

The Role of Relevance in Medical Education Research

Miller, Karen Hughes, PhD, MEd; O’Brien, Bridget, PhD; Karani, Reena, MD, MHPE

doi: 10.1097/ACM.0000000000002385
Foreword
Free

K.H. Miller is immediate past chair, Research in Medical Education Program Planning Committee, and recently retired associate professor of graduate medical education, University of Louisville School of Medicine, Louisville, Kentucky.

B. O’Brien is incoming chair, Research in Medical Education Program Planning Committee, and associate professor, Department of Medicine, Division of General Internal Medicine, University of California, San Francisco, San Francisco, California.

R. Karani is 2018 chair, Research in Medical Education Program Planning Committee, and senior associate dean for undergraduate medical education and curricular affairs and director, Institute for Medical Education, Icahn School of Medicine at Mount Sinai, New York, New York.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

On behalf of the Research in Medical Education (RIME) Program Planning Committee, welcome to the 57th annual Association of American Medical Colleges (AAMC) RIME program. We hope you will benefit from reading the research papers, reviews, Foreword, and Commentary included in this RIME supplement to Academic Medicine and from participating in the RIME sponsored sessions during Learn Serve Lead 2018. Our sincere thanks to the medical education researchers who submitted manuscripts for consideration, and to the 52 reviewers who participated in the peer review process. Your work is the substance of RIME, and each year we look forward to seeing what new ideas come our way. And thanks to the AAMC staff who provide so much more than administrative support. Their insightful perspective informs and guides us.

Each year, as the committee conceptualizes the Foreword to the RIME supplement, we consider both our past few years’ activities and our future direction in terms of work submitted, work accepted, themes of our invited addresses during Learn Serve Lead; and, perhaps a little selfishly, our own desired directions for the future of medical education research. We hope that our Forewords provide useful guidance in medical education research, especially to medical trainees and early-career faculty. The 2016 RIME Foreword outlined the elements of a successful RIME submission,1 while the 2017 Foreword explored the roles of reviewers and authors.2 This year’s Foreword examines the role of “relevance” in medical education research to encourage us all to consider a wider range of research topics with the potential to inform and improve medical education across the continuum and future health care delivery.

Back to Top | Article Outline

What Is Relevance?

The traditional measures of value for medical education research include the clarity of the research question, the appropriateness of the study design, the rigor of the methodology (quantitative, qualitative, and/or review), the strength of the evidence, and the generalizability/transferability of the findings.3,4 These elements can be evaluated, and many resources5–10 provide a wealth of information to guide design and evaluation of studies. Many reviewer guidelines and discussions of good research also mention “relevance”—a more elusive and contested construct.3,4

Relevance is situated in time and place—it depends on the political, sociocultural, historical, technological, and perhaps even epistemological context. Some argue that education is an applied field in which research is grounded in “problems of practice.” From this perspective, relevance is based on the extent to which research addresses and yields findings that are useful for resolving “problems of practice.”11,12 Of course, deciding what constitutes a useful question or findings depends on who you ask. A dean, a student or resident, a clinician–educator, a researcher, a patient, or a member of an accrediting body will likely have quite different views of relevance and value.

In light of these perspectives, how do we as a research community think about “relevance” and define criteria that guide our design and evaluation of research in a productive way?

Back to Top | Article Outline

Conceptualizing “Relevance”

The 2004 RIME Committee was among the first to explore the construct of “relevance” in conjunction with “quality” of medical education research.4 They characterized relevance as studies that address “the ongoing concerns of the research in the medical education community” and that “hold implications for the enduring concern of improving either practices of medical education and/or medicine.”

Frequent RIME contributor Larry Gruppen13 argued as early as 1990 that physicians in training learn best when they see the immediate relevance of content: “Relevance and practicality are central.” Although he was writing in the Bulletin of the Medical Library Association for librarians, the message also applies to those of us who develop that literature. We must not forget to explicitly note the relevance and practicality of our work.

When a researcher views a question as important, the researcher often assumes the question will be important to a wider audience. But, as George Gershwin said, “It ain’t necessarily so!” Even if the results of a study are generalizable (or, in the case of qualitative studies, the concepts or theories transferable),14,15 researchers cannot assume that the individuals or organizations they believe will benefit from the findings will see the relevance and be able to make changes. Along with the other elements of good research design, researchers must present a strong argument why and to whom their study is relevant.16–18

A pragmatic approach to improving relevance is to acknowledge multiple types of relevance and to encourage authors to be explicit about ways in which their work is relevant (or useful) to others. For example, a scoping review may not offer specific solutions to a practical problem, but it may bring clarity to a complex topic and identify problems that warrant further research and/or innovative approaches in practice. Such work could be relevant to both researchers and clinician–educators. By contrast, an evaluation of a curricular reform may offer solutions to a problem faced by a single institution, but a stronger case for relevance should indicate the value of the reform and the evaluation findings to broader problems in the field. Such work might be useful to educational leaders or clinician–educators seeking to address similar problems in their own institutions. Or it might be useful to researchers who wish to delve more deeply into questions about why the reform produced certain outcomes.

Authors can also use their choice of journals as an opportunity to maximize relevance. Each journal has a different mission and audience. The journal in which work is published helps establish relevance by placing work in context, relating it to other published work, and reaching readers who will likely know how to use or apply the study findings.

Often, particularly when exploring issues of social or societal relevance, researchers develop true passion for an issue. While passion can drive the desire to research certain topics, one’s stance as a researcher must remain critical and open to all perspectives and findings. Soapbox rhetoric and linear regression are an awkward combination, so how and where in scholarly writing can researchers make a clear and compelling case for the relevance of their research without appearing biased?

Relevance needs to be argued differently depending on the intent of the manuscript. For example, research studies are a different genre from commentaries and editorials and are written (and cited) differently. In commentaries and editorials, authors can argue for relevance using a larger toolbox including both scholarly work and more community-based sources. They must commit to using reputable data, but descriptive and reflective language is also appropriate. They can express themselves and their perspectives (biases?) because their goal is persuasion.

In research studies, authors wear a different hat. While it is essential to state the case for relevance, they must be rigorous and systematic to minimize bias so as not to compromise the validity or trustworthiness of the research and acknowledge the perspectives they bring to the study. There are several places in a well-written study where this can be done. This first is in the introduction where authors place their work in context and describe the problem or gap that generated the research question.19 Context may include citations of related studies but may also include editorials or commentaries when identified as such. Citing an editorial or commentary from a well-respected journal, especially one that has made a call for the research being presented, can help build the argument for the relevance of the research.

The research problem must be observable and measurable, and, while stated in unbiased language, it may certainly imply a need for change. For example, “many primary care physicians in the United States feel unprepared to provide care to transgender patients.”20,21 There are tremendous social implications behind this statement, but the research finding itself is expressed in unbiased terms.

The Discussion section also provides authors opportunities to help readers see the relevance of their work. Once outcomes are stated, authors can compare their findings with outcomes of studies in other contexts or disciplines. This approach not only supports the argument for relevance but also shows that many problems faced in medical education are not unique.22,23

Many methodological approaches offer opportunities for providing evidence of relevance. As medical education research ventures into new content areas, members of the medical education research community can draw on a variety of methodologies to study pressing and emerging problems they know well from their own observations and experiences. For example, in phenomenological approaches, the researcher explores people’s lived experiences of a situation or event (a phenomenon).24 This methodological approach can be a highly effective way of capturing trainee and faculty insights in the classroom and clinical learning environments. Yet another example is epidemiological research which has both medical and behavioral applications and thus can be applied to current issues such as gun violence25 and opioid addition and their importance in medical education.26 West et al1 reminded us in the 2016 RIME Foreword that methodology can be innovative, but must be well referenced so readers understand not only what you did, but why you did it. And your argument for why is an additional opportunity to remind readers of the relevance of your work.

Back to Top | Article Outline

Conclusions

Medical education research is maturing. The growing community of trainee and faculty researchers benefits from an increasing body of knowledge on which their work can build. Medical education research has addressed many pragmatic questions common to medical educators and provided evidence to support good curricular decision making. It is now time to dive deeper into the more difficult questions—those that deal with the social and psychological impact of medical education on trainees, faculty, and ultimately on patients and society as a whole. While it is true that “relevance” is situational, that in no way reduces its importance. The work of medical education researchers resides at the intersection of the postsecondary education system and the health care system. Both systems have tremendous complexity.

As we broaden our scope of work and our methodologies, we must also be innovative in expanding our audience. How might administrators, politicians, and policy makers benefit from our findings? How might academicians in other disciplines become our research partners? How can we make our work more accessible and comprehensible to a variety of stakeholders and audiences? We argue that including clear statements of relevance (both immediate and potential) will increase the likelihood that our work will be found, and outcomes applied. The “so what?” factor that we were taught as novice researchers remains essential.

Back to Top | Article Outline

Gratitude

We thank several individuals and groups who helped make this year’s RIME program a reality. First, we thank the dedicated AAMC staff—Nesha Brown, Steve McKenzie, and Kate McOwen—who worked tirelessly to make the RIME program and supplement happen. We also thank the 2018 RIME Program Planning Committee: Lara Varpio, Tanya Horsley, Win May, Jeanne Farman, Douglas Larson, Zareen Zaidi, and Yoon Soo Park. It is an honor and privilege to work with such a talented group of individuals who are so devoted to promoting the art and science of medical education. Finally, we thank Dr. David Sklar and the editorial team of Academic Medicine. We sincerely appreciate their continued support of medical education research and the RIME program.

Back to Top | Article Outline

References

1. West DC, Miller KH, Artino AR Jr.. Foreword: Characteristics of RIME papers that make the cut. Acad Med. 2016;91(11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 55th Annual Research in Medical Education Sessions):Si–Siii.
2. Miller KH, West DC, Karani R. Foreword: Sticks and stones: Moving toward a more productive peer-review process for authors and reviewers. Acad Med. 2017;92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):Si–Siii.
3. Durning SJ, Carline JD. Review Criteria for Research Manuscripts. 2015. 2nd ed. Washington, DC: Association of American Medical Colleges; https://www.aamc.org/download/484320/data/reviewcriteriaforresearchmanuscripts.pdf. Accessed July 31, 2018.
4. Shea JA, Arnold L, Mann KV. A RIME perspective on the quality and relevance of current and future medical education research. Acad Med. 2004;79:931–938.
5. Genn JM. AMEE medical education guide no. 23 (part 1): Curriculum, environment, climate, quality and change in medical education—A unifying perspective. Med Teach. 2001;23:337–344.
6. Genn JM. AMEE medical education guide no. 23 (part 2): Curriculum, environment, climate, quality and change in medical education—A unifying perspective. Med Teach. 2001;23:445–454.
7. Ringsted C, Hodges B, Scherpbier A. “The research compass”: An introduction to research in medical education: AMEE guide no. 56. Med Teach. 2011;33:695–709.
8. The EQUATOR Network. Enhancing the quality and transparency of education research. Reporting guidelines. https://www.equator-network.org/reporting-guidelines. Accessed July 23, 2018.
9. Picho K, Artino AR Jr.. 7 deadly sins in educational research. J Grad Med Educ. 2016;8:483–487.
10. O’Brien BC, Balmer DF, Maggio LA. Finding our way through shades of gray: 6 virtues to guide researchers in planning, conducting, and writing up research. J Grad Med Educ. 2017;9:555–559.
11. Albert M, Hodges B, Regehr G. Research in medical education: Balancing service and science. Adv Health Sci Educ. 2007;12:103–115.
12. Cook DA, Bordage G, Schmidt HG. Description, justification and clarification: A framework for classifying the purposes of research in medical education. Med Educ. 2008;42:128–133.
13. Gruppen LD. Physician information seeking: Improving relevance through research. Bull Med Libr Assoc. 1990;78:165–172.
14. Baker L, Phelan S, Snelgrove R, Varpio L, Maggi J, Ng S. Recognizing and responding to ethically important moments in qualitative research. J Grad Med Educ. 2016;8:607–608.
15. Golafshani N. Understanding reliability and validity in qualitative research. Qual Rep. 2003;8(4):597–607. http://nsuworks.nova.edu/cgi/viewcontent.cgi?article=1870&context=tqr. Accessed July 31, 2017.
16. Hojat M, Xu G. A visitor’s guide to effect sizes—Statistical significance versus practical (clinical) importance of research findings. Adv Health Sci Educ. 2004;9(3):241–249.
17. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: Issues in external validation and translation methodology. Eval Health Prof. 2006;29:126–153.
18. Panda A, Gupta RK. Making academic research more relevant: A few suggestions. IIMB Manag Rev. 2014;26(3):156–169.
19. Lingard L. Joining a conversation: The problem/gap/hook heuristic. Perspect Med Educ. 2015;4:252–253.
20. Kitts RL. Barriers to optimal care between physicians and lesbian, gay, bisexual, transgender, and questioning adolescent patients. J Homosex. 2010;57:730–747.
21. Snelgrove JW, Jasudavisius AM, Rowe BW, Head EM, Bauer GR. “Completely out-at-sea” with “two-gender medicine”: A qualitative analysis of physician-side barriers to providing healthcare for transgender patients. BMC Health Serv Res. 2012;12:110.
22. Sutherland WJ, Goulson D, Potts SG, Dicks LV. Quantifying the impact and relevance of scientific research. PLoS One. 2011;6:e27537.
23. Chastonay P, Brenner E, Peel S, Guilbert FF. The need for more efficacy and relevance in medical education. Med Educ. 1996;30:235–238.
24. Ajjawi R, Higgs J. Using hermeneutic phenomenology to investigate how experienced practitioners learn to communicate clinical reasoning. Qual Rep. 2007;12(4):612–638.
25. Scott S. Using Cellular Automata to Model the Gun Violence Epidemic in Chicago, IL. https://core.ac.uk/download/pdf/48842056.pdf. Accessed August 20, 2018.
26. Kolodny A, Courtwright DT, Hwang CS, et al. The prescription opioid and heroin crisis: A public health approach to an epidemic of addiction. Annu Rev Public Health. 2015;36:559–574.
© 2018 by the Association of American Medical Colleges