Failure to engage in the responsible conduct of research (RCR), or failure to conduct research with integrity, is a concern across disciplines and with good reason. A comprehensive study published in 20091 indicated that approximately 2% of scientists report falsifying or modifying data, and that 34% have admitted to other forms of research misconduct. In 2011, Diederik Stapel, a Dutch psychologist, was denounced as a fraudster for fabricating and manipulating research data disseminated in over 50 publications.2 Stapel’s research misconduct led to his dismissal from the university; had an impact on multiple careers, including those of his graduate students; and damaged the public’s trust in science. Research misconduct is defined as the “fabrication, falsification, or plagiarism in proposing, performing or reviewing research, or in reporting research results.”3 While instances of research misconduct are clearly the most serious of violations, the more prevalent form of irresponsible research conduct is probably engaging in questionable research practices (QRPs).4
QRPs are generally described in the psychology literature as the selective reporting of variables, rounding of P values, and failure to disclose study conditions.4 While we agree that these research practices are indeed questionable, we propose that QRPs also encompass a broader spectrum of irresponsible research conduct. Thus, we define QRPs as poor data practices and inadequate data management, inappropriate research procedures—including questionable procedures for obtaining informed consent— insufficient respect and care for participants in the research, improper research design, carelessness in observation and analysis, suboptimal trainee and mentor partnerships, unsuitable authorship or publishing practices, and derelictions in reviewing and editing.5
QRPs have been recognized as an important issue broadly in the biomedical and social sciences4; however, they have received less attention in medical education research. Although outright fraud may seem relatively unlikely and is obviously wrong and unthinkable to most medical education researchers, instances of QRPs are not as unlikely and certainly are not as clear-cut.
Instead, when it comes to QRPs, medical education researchers often find themselves faced with dilemmas that have no simple solutions or perfect outcomes. Such situations can be characterized as colored in shades of gray, not black and white. Also, researchers may not even be aware that they are engaging in QRPs because of a lack of training and knowledge about what constitutes a QRP. QRPs can lead to publication results that are dangerous for use in patient care and ineffective for education, misinform policy, result in a scientific record that is unable to be replicated or built upon, and waste funds that could more appropriately support other research endeavors.3,6
With these issues in mind, we wrote this Perspective to raise awareness of RCR in medical education research and to call the community to action before QRPs negatively affect our field (if they have not done so already). To frame this Perspective, we introduce an example of a research dilemma that could easily happen in medical education. We next describe why QRPs occur so often, in general, and highlight potential vulnerabilities particular to medical education research. We then recommend five approaches to promoting responsible research conduct in medical education drawn from tactics used in other fields. These approaches are intended to inform research training and culture. We propose them in relation to the growth of graduate programs,7 fellowships,8 and faculty academies9 in medical education, which underscores the timeliness of this call to action.
What Do QRPs Look Like?
Consider the following dilemma:
For his dissertation, doctoral student Rogers measured the relation between first-year students’ metacognitive skills and their grades. He divided the scores on the metacognitive skills questionnaire as originally planned, but was disappointed that analyses of the results did not show a significant relation. He discussed the findings with his supervisor. His supervisor tried some alternatives for dividing the questionnaire scores until he found one that produced statistically significant associations with the grade variable. The supervisor advised student Rogers to write up the study with this alternative score division. What should the student do?
This dilemma is just one of many that medical education researchers may face, often on a fairly regular basis. Although the scenario itself has several interpretations, at the very least, the supervisor’s manipulation of the questionnaire results and testing of several statistical models likely represents a fairly common analysis technique often referred to as P hacking. For more examples of such dilemmas, see Box 1. Each of these dilemmas crosses several RCR issues (e.g., authorship and publication practices, data ownership, data collection and analysis, collaboration management) that can arise in medical education. As readers will note, there are no easy or perfect answers for how a researcher should proceed in these situations. Furthermore, there are limited guidelines and resources to consult when dealing with such dilemmas.
Box 1Potential Dilemmas in Medical Education Researcha
Dr. Flynn teaches a course for clerkship students on medical ethics. Early in the term, she learns that her students are sharing incidents they consider ethics violations on Twitter using #ethicsfailXU. Dr. Flynn joins Twitter as @ethicsmd and begins following her students’ tweets out of curiosity and to improve her course. After the term, Dr. Flynn analyzes the students’ tweets and publishes a related journal article. The article does not identify individual students, but does include verbatim tweets. Shortly after the article is published, an outraged student tweets: “Monitoring and publishing student tweets! WHAT??? Creepy example of #ethicsfailXU by @ethicsmd .” How should Dr. Flynn respond?
Potential responsible conduct of research issue:
- Here, the issue is who owns the contested tweets?
- Dr. Flynn could acknowledge that because of this uncertainty about data ownership, it would have been prudent for her to have obtained permission from the students to use their tweets, and if there had been any objection to a particular tweet being quoted, to either not use it or to revise it sufficiently so it could not be identified. She could then move on to a wider discussion of research dilemmas and infractions, and how, when these occur, these must be openly aired by all parties if possible to help prevent future problems.
- Dr. Flynn could explain that obtaining permission from the students to use their tweets is an example of a mechanism for facilitating responsibly conducted research. She could then move on to a wider discussion of various mechanisms of this type and about the public nature of content posted to the Web.
PhD student Green circulates a final draft of a manuscript to her research advisors for their sign-off. She receives back a comment that recommends adding Professor Karr, a senior professor in her department, as a coauthor. Her primary advisor suggests that it would garner departmental goodwill and that Professor Karr’s reputation may help get the manuscript published. Since Professor Karr has not contributed to the article, PhD student Green feels uncomfortable with the inclusion but is unsure how to proceed.
Potential responsible conduct of research issues:
- Here, the issue is what qualifies someone for authorship? In addition, there are important power and control issues that relate to the trainee–mentor partnership.
- Fortunately, the rules regarding authorship are fairly straightforward, although as this vignette suggests, researchers may not always strictly adhere to those requirements. Most medical education journals follow the International Committee of Medical Journal Editors’ (ICMJE’s) Recommendations for the Conduct, Reporting, Editing and Publication of Scholarly Work in Medical Journals (http://www.icmje.org/icmje-recommendations.pdf) for determining authorship. According to these recommendations, authorship is based on (1) making substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content; (3) giving final approval of the version to be published; and (4) agreeing to be accountable for all aspects of the work. Authors must meet all of these conditions. Thus, it is clear that Professor Karr does not meet the criteria for authorship defined above.
- The challenge in this scenario, however, is the unequal power distribution between the PhD student and her primary advisor. The advisor has the power; the student does not. Nonetheless, it is incumbent upon the student to voice her concerns to the advisor. If the advisor still insists, the student should approach a trusted colleague or ombudsman (i.e., an independent, student advocate), who can be an advocate for her. Importantly, the institution should have policies in place for handing such situations; if necessary, they should protect the student as a whistleblower. Finally, the institution should consider handling the situation at the lowest possible level, as well as educating the advisor by reiterating his role and empowering him as a research role model.
Dr. Bond advises doctoral candidates from around the globe in health professions education and serves as a coauthor on the manuscripts produced. One of his students plans to publish a qualitative study, based on interviews in the student’s native language. Dr. Bond is unable to speak or read this language. The student presents to Dr. Bond a list of themes in English that he has identified from his transcript analysis. How should Dr. Bond proceed?
Potential responsible conduct of research issues:
- Here, the issue is how should trainees and advisors effectively and ethically collaborate? Additionally, there are issues related to qualifications for authorship.
- Dr. Bond could engage the student in a conversation about the importance of data transparency and state that, based on ICMJE guidelines, it is important for Dr. Bond, as a coauthor, to substantially contribute to the analysis and interpretation of the data. In the conversation, he could request that the student provide a sample of his interview transcripts, translated.
- In the future, to establish mechanisms for facilitating responsibly conducted research, Dr. Bond could work with his department to craft policies related to data sharing between advisors and students. He could also be an advocate for providing students resources for data translation if their data are not collected in a language readily understandable to the advisor.
aThe dilemmas presented above are only a few of the many that medical education researchers face, often frequently. There are no easy or perfect answers for how a researcher should proceed in these situations. Furthermore, there are limited guidelines and resources to consult when dealing with such dilemmas.
Why Do Researchers Sometimes Engage in QRPs?
There are a variety of reasons that researchers, including those in medical education, may engage in QRPs. First, researchers are under incredible pressure to publish original research in an effort to earn promotion and tenure. Additionally, there is pressure to publish innovative research with positive, statistically significant results.10 Moreover, researchers often are unaware of the “rules” for RCR because of a lack of knowledge of what constitutes a QRP. In medical education, this problem is especially relevant because many researchers have received their training in other disciplines (e.g., biomedicine, psychology). Furthermore, in the United States, much of RCR training is driven by federal funding mandates. However, besides very basic (typically online) institutional review board (IRB) training (see, for example, the Collaborative Institutional Training Initiative11), RCR training is not generally compulsory for unfunded researchers and, in many cases, is required only for graduate students.12 Therefore, it is quite possible that faculty and students engaging in nonfederally funded research, including medical education research, may not receive RCR training and thus may be ignorant of the issues therein.
While researchers in any discipline might engage in QRPs, medical educators should consider certain specific characteristics of medical education and how these characteristics may raise the susceptibility of its researchers. To start, medical education is a multidisciplinary field that includes scholars from a variety of backgrounds (e.g., clinical medicine, psychology, education, anthropology) and with varying levels of experience in conducting medical education research.13 For example, a U.S. clinician scientist working on his first medical education project may be unaware that it is a questionable practice to randomly query student evaluation data—looking for a particular, statistically significant finding—without first formulating a research question and obtaining IRB approval.
Additionally, interdisciplinary (and often global) sharing and adopting of methods, methodologies, and research traditions are increasingly common in medical education, making it quite possible that researchers are unaware of methodological requirements and responsible research practices in other disciplines and countries, which may differ from their own.14,15 For example, there are differences in how disciplines determine the inclusion of an author on a manuscript or decide author order. What is more, there are differences in how disciplines define the meaning and value of specific authorship positions. For instance, in medical education, the last position is typically the “senior author,” whereas in other fields, like psychology and education, no such senior author position is acknowledged. Although not immediately apparent to some, lack of knowledge regarding author order may be related to other unsuitable author practices, like “honorary authorship” for a researcher who has not met established authorship criteria.16
Finally, learners may become lured into QRPs as they follow the examples set by senior faculty or in response to the power dynamics between graduate students and their supervisors (as in the dilemma described above), which puts students at a significant disadvantage and may make them hesitant to speak up. In the case of Stapel,2 a researcher who coauthored many articles with his graduate students, it was discovered later that his trainees often did not raise concerns when Stapel independently ran experiments for them.17 In medical education, a field with a growing number of graduate programs,7 this power dynamic is an important concern. Furthermore, even if graduate students or fellows have been exposed to research ethics training in their roles as physicians or clinical researchers, they may not necessarily make the connection or have considered how to transfer that knowledge over to medical education research.18
These examples of QRPs demonstrate a few critical points. First, QRPs often are not the result of an unethical researcher looking to make his or her scholarly mark on the field. Instead, like most complex social phenomena, irresponsible research conduct occurs from the multifactorial effects of both personal factors (e.g., knowledge, beliefs, attitudes) and environmental or contextual factors (e.g., social norms, power dynamics, intuitional policies). Thus, there is a need for both individual and systemic approaches to fostering RCR. Discussions about and efforts to improve RCR have been increasingly promoted in biomedicine and psychology. However, on the basis of our experiences as medical education researchers, journal editors, and faculty members in graduate programs, RCR efforts in medical education have been less robust, often focusing solely on the individual, as opposed to focusing on the individual functioning in a complex learning and health care environment.19
What Can Medical Education Do?
As medical education continues to establish itself as a discipline, including through the growth of graduate degree and other educational programs, it is necessary to amplify the conversation about RCR and ensure that our community members are prepared to act as responsible researchers and, moreover, that our systems and institutional policies support them. An instinctive way to guard against QRPs might be to create regulations and/or require standardization of research practices. However, this approach has been tried with limited success in other fields.6 Additionally, the regulations approach often focuses on producing a list of “what not to do” instead of providing researchers with advice for how to deal with difficult, complex situations and dilemmas when they occur.
Therefore, we suggest that efforts in research, including medical education research, focus on facilitating a change in the culture of research, and we propose a (likely noncomprehensive) set of potential approaches to foster a culture of RCR in medical education. To facilitate this culture change, we recommend focusing on the growing training programs and faculty academies in medical education. The time is ripe to shape these programs and to ensure that RCR training is an integral component for both students and faculty. For example, international efforts to improve the quality of master’s programs in health professions education would do well to include knowledge and skill in RCR issues as a foundational competency. What is more, such efforts might also encourage programs to have in place structures and procedures for dealing with RCR issues. Even so, such efforts cannot stand alone. As mentioned above, educational efforts in other fields, such as the biomedical sciences, have demonstrated that training and additional requirements alone do not work.20 Instead, educators must create a research culture within their programs that supports RCR efforts and makes explicit the system effects that ultimately influence researcher behavior. Therefore, we encourage the integration of the following five approaches within medical education research training initiatives.
Empower research mentors as role models
Mentors are important for the socialization of junior researchers into RCR.21 They can support this socialization in a variety of ways. First, they can teach or discuss RCR with their students (e.g., by explaining concepts like the role of ethical review committees and, if available, bringing to the attention of students the code of conduct for researchers and issues relating to authorship, transparency, and other areas). Of course, it is one thing for mentors to explain RCR to their students, but it is another, much more important task for them to influence their students by what they do in their daily research practices. In other words, a mentor is first and foremost a role model. Supervisors, therefore, need to behave as responsible researchers. For example, this could mean not accepting coauthorship on a mentee’s article without first contributing in a meaningful way to the article. The effect of role modeling becomes even stronger when role models make explicit the motives behind the choices they have made.
Mentors can also support students’ socialization into RCR by helping students reflect on their research behavior during progress meetings. The mentor can initiate this reflection by asking questions about motives for, and confronting students with consequences of, planned behavior.22 Last, it is not only the students’ supervisors who can provide mentorship. Other faculty with whom students have relationships, as well as peers (i.e., other graduate students), offer the potential for research mentoring. Peers and faculty outside the supervisory team typically have little to no power relationship with students, which could make it easier for them to discuss with their peers more sensitive ethical topics.
Openly air RCR dilemmas and infractions
An open and honest research culture should include an environment in which dilemmas and issues are openly discussed in research teams early and often in the research process. Such an environment would encourage and provide trainees and faculty a safe space to report dilemmas and infractions or to integrate the concept of a research “morbidity and mortality” conference. Following the example set by the Committee on Publication Ethics (COPE),23 we have provided three dilemmas in Box 1, which could be used to facilitate ethical conversations, could be integrated into medical education training programs, and could be added to by researchers as they experience or learn of additional dilemmas.
In addition to openly airing RCR dilemmas, it is incumbent on institutions and training programs to both empower and protect whistleblowers. As Kornfeld24 has argued,
because the total prevention of research misconduct is impossible, the scientific community must depend on whistleblowers to minimize the presence and/or persistence of flawed data in the scientific literature.
At the same time, however, we believe in the importance of dealing with whistleblower allegations at the lowest level possible within an organization, and doing so with due diligence. Not all RCR dilemmas or infractions warrant punitive action, and we suspect that awareness of most infractions likely does not need to go beyond departmental walls. Thus, research and training institutions are obligated to create mechanisms that protect the interests of all those involved in accusations of research misconduct and to communicate the existence of such mechanisms to their community.24 Much like in the U.S. legal system, those accused must be treated fairly and considered innocent until proven guilty, not the other way around. Academic careers take a lifetime to build and only one mishandled accusation to destroy. What is more, we believe that by not escalating all alleged infractions to the highest level, institutions further encourage the openness and frank discussions recommended in the paragraph above.
Establish mechanisms for facilitating responsibly conducted research
Institutions must ensure mechanisms are in place to facilitate researchers’ overcoming barriers to responsibly conducting medical education research. For example, medical education researchers have raised concerns about the review of medical education research protocols by IRBs whose members may be unfamiliar with or inexperienced in reviewing these types of protocols.25 To mitigate researcher concerns, medical educators at Duke University designed a medical-education-specific template for researchers to submit medical education protocols to their IRBs.26 Going a step further, the Netherlands Association for Medical Education has created a nationwide ethics review board specifically intended to review medical education protocols for its members.27
Additionally, because current IRB processes are primarily meant to protect research subjects rather than to provide oversight of the quality of the research, we also suggest that institutions consider supplemental supports for RCR and to safeguard research quality. Potential approaches might include forming groups or peer networks in which members would have opportunities to present and discuss RCR in relation to ongoing and future projects and within the published literature (e.g., an RCR journal club). Institutions might also consider potential RCR issues in the formal assessment of research proposals, including thesis and dissertation proposals.
Reward responsible researchers
Medical education researchers should be rewarded for conducting systematic and transparent research. To encourage researchers, programs might consider adapting promotion and tenure guidelines to specifically welcome types of publications that demonstrate responsible research practices, such as the publication of research protocols for education projects and replication studies. Research practices in both of these types encourage rigorous, programmatic approaches to science, which in turn further develop and strengthen research methods and theories.
In addition to encouraging such practices, research rewards could be tied to funding. For example, the Netherlands Organization for Scientific Research, that nation’s largest funder, recently pledged several million euros to support replication studies.28 Other institutions and agencies that provide funding for medical education research could follow suit, encouraging researchers to not only undertake replication studies but also partake in other responsible research practices, such as providing open access to study materials, depositing their data for potential reanalysis by other researchers, preregistering their studies, and publishing research with negative findings.29 Of course, this last suggestion—publishing negative findings—also requires a commitment from journal editors. But as journals continue to move away from paper issues and toward online publication, page limits are potentially becoming less of a factor, thereby making space for negative studies.
The Need for Greater Understanding of RCR
The above five approaches are suggestions for how medical education might move forward in creating a culture of RCR. We are aware that at this time there is limited evidence that these approaches would work. Furthermore, we recognize the need to gauge the prevalence of QRPs in medical education research. We hypothesize that QRPs are a serious problem, based on incidents reported by journal editors19,30,31 and literature from other domains,2,32 but currently we do not have a concrete sense for the magnitude or nature of the problem. This gap in our understanding makes it difficult to know what approaches to take, and it suggests the need for future research that assesses the nature (e.g., prevalence, causes, effects) of QRPs in medical education.33 An understanding of key RCR issues would facilitate clearer thinking about future approaches and may help delineate the most appropriate behavior or systems change models.
In this Perspective, our purpose was to raise awareness of the need for responsible research conduct in medical education research. In addition, we have called on our community to take action before QRPs become a major issue in the field (assuming they have not already). To help them answer this call, we proposed five approaches to improve the research culture and promote RCR within graduate programs, fellowships, and faculty academies in medical education, in light of their growth in recent years. We hope, however, that this Perspective is only the beginning of the conversation. Furthermore, we encourage medical education researchers to think creatively about solutions to the challenges we face and to act together as an international community to avoid wasting research efforts, damaging careers, and stunting medical education research through QRPs.
1. Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS One. 2009;4:e5738.
3. Steneck NH. Office of Research Integrity: Introduction to the Responsible Conduct of Research. 2007.Bethesda, MD: Government Printing Office
4. John LK, Loewenstein G, Prelec D. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychol Sci. 2012;23:524–532.
5. ALLEA Permanent Working Group on Science and Ethics. The European Code of Conduct for Research Integrity, Revised Edition. 2017.Berlin, Germany: ALLEA–All European Academies
7. Tekian A, Harris I. Preparing health professions education leaders worldwide: A description of masters-level programs. Med Teach. 2012;34:52–58.
8. Thompson BM, Searle NS, Gruppen LD, Hatem CJ, Nelson EA. A national survey of medical education fellowships. Med Educ Online. 2011;16. doi: 10.3402/meo.v16i0.5642.
9. Searle NS, Thompson BM, Friedland JA, et al. The prevalence and practice of academies of medical educators: A survey of U.S. medical schools. Acad Med. 2010;85:48–56.
10. Martinson BC, Anderson MS, de Vries R. Scientists behaving badly. Nature. 2005;435:737–738.
11. Braunschweiger P, Goodman KW. The CITI program: An international online resource for education in human subjects protection and the responsible conduct of research. Acad Med. 2007;82:861–864.
12. Resnik DB, Dinse GE. Do U.S. research institutions meet or exceed federal mandates for instruction in responsible conduct of research? A national survey. Acad Med. 2012;87:1237–1242.
13. Peeraer G, Stalmeijer RE. Research fraud and its combat: What to do in the case of qualitative research. Med Educ. 2014;48:333–334.
14. Bulger RE, Heitman E. Expanding responsible conduct of research instruction across the university. Acad Med. 2007;82:876–878.
15. Steneck NH, Bulger RE. The history, purpose, and future of instruction in the responsible conduct of research. Acad Med. 2007;82:829–834.
18. Blanchard RD, Artino AR Jr, Visintainer PF. Applying clinical research skills to conduct education research: Important recommendations for success. J Grad Med Educ. 2014;6:619–622.
19. ten Cate O, Brewster D, Cruess R, et al. Research fraud and its combat: What can a journal do? Med Educ. 2013;47:638–640.
20. Kalichman M. Responsible conduct of research education (what, why, and does it work). Acad Med. 2016;91:e10.
21. Fisher CB, Fried AL, Feldman LG. Graduate socialization in the responsible conduct of research: A national survey on the research ethics training experiences of psychology doctoral students. Ethics Behav. 2009;19:496–518.
22. Driessen E, Overeem K. Walsh K. Mentoring. In: Oxford Textbook of Medical Education. 2013.London, UK: Oxford University Press
24. Kornfeld DS. Perspective: Research misconduct: The search for a remedy. Acad Med. 2012;87:877–882.
25. Dyrbye LN, Thomas MR, Papp KK, Durning SJ. Clinician educators’ experiences with institutional review boards: Results of a national survey. Acad Med. 2008;83:590–595.
26. DeMeo SD, Nagler A, Heflin MT. Development of a health professions education research-specific institutional review board template. Acad Med. 2016;91:229–232.
27. Eikelboom JI, ten Cate OT, Jaarsma D, Raat JA, Schuwirth L, van Delden JJ. A framework for the ethics review of education research. Med Educ. 2012;46:731–733.
29. Munafo MR, Nosek B, Bishop D, et al. A manifesto for reproducible science. Nat Hum Behav. 2017;1. doi: 10.1038/s41562-016-0021.
30. Brice J, Bligh J, Bordage G, et al. Publishing ethics in medical education journals. Acad Med. 2009;84(10 suppl):S132–S134.
31. Norman G. Data dredging, salami-slicing, and other successful strategies to ensure rejection: Twelve tips on how to not get your paper published. Adv Health Sci Educ Theory Pract. 2014;19:1–5.
32. Ioannidis JP. How to make more published research true. PLoS Med. 2014;11:e1001747.
33. Fanelli D, Costas R, Larivière V. Misconduct policies, academic culture and career stage, not gender or pressures to publish, affect scientific integrity. PLoS One. 2015;10:e0127556.