Secondary Logo

Journal Logo

Articles

Gamification in Action: Theoretical and Practical Considerations for Medical Educators

Rutledge, Chrystal MD; Walsh, Catharine M. MD, MEd, PhD, FRCPC; Swinger, Nathan MD; Auerbach, Marc MD, MSc; Castro, Danny DO, MEd; Dewan, Maya MD, MPH; Khattab, Mona MD; Rake, Alyssa MD; Harwayne-Gidansky, Ilana MD; Raymond, Tia T. MD; Maa, Tensing MD; Chang, Todd P. MD, MAcM; for the Quality Cardiopulmonary Resuscitation (QCPR) leaderboard investigators of the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE)

Author Information
doi: 10.1097/ACM.0000000000002183

Abstract

Gamification involves the application of game design elements (conceptual building blocks integral to building successful games) to traditionally nongame contexts.1–3 Gamification started to become a worldwide trend around 2010.4,5 The notion of gamification began with the idea that because video games can capture significant attention and engagement for long periods of time, the application of game design elements to nongame phenomena should also increase attention and engagement.1 Game design elements can be as simple as badges, leaderboards, and points or as complex as evoked emotion, narratives, and competition.6 (See Table 1 for definitions and examples of these design elements.)

T1
Table 1:
Examples of Game Design Elements Commonly Applied in Medical Educationa

Over the last decade, numerous companies have applied gamification to incentivize nongame activities. For example, Foursquare, a popular search-and-discovery service app, rewards users for visiting businesses. Nike developed Nike+, a social running app that awards achievement badges for completing running challenges, to encourage healthy behavior change. A more innovative use of this design technique is from a recycling company, Recyclebank, which rewards recycling by providing points redeemable for purchases, to promote sustainable living practices. Companies across a diverse range of industries report applying gamification to foster engagement, behavior change, friendly competition, and collaboration.7,8

The maturation and expansion of gamification have led to further research into engagement and motivation, whose principles have subsequently been applied to more purposeful and/or productive activities.1,8 For example, discovery of a complex HIV protein structure was achieved through FoldIt, an online puzzle interface that enables players to “solve” protein-folding puzzles.9 Game design elements, including competition through points and leaderboards, led to significant public engagement and resulted in players solving this long-standing scientific problem in 3 weeks, a feat that scientists had struggled to achieve for 15 years. More recently, gamification has been applied within the field of education, particularly at the university level, to increase learner engagement, motivation, and retention.8,10,11 Despite its growing popularity, emerging evidence within the educational context continues to show mixed results.1,8,10

Medical education has only recently implemented gamification strategies to engage the millennial learner. Gamification strategies are used by a variety of specialties, from surgery to internal medicine and radiology.12–14 For example, thoracic surgery residencies have used gamification as a way to increase residents’ use of simulation-based training.15 The application of gamification is attractive because of its potential to improve adult learner engagement and motivation, but its theoretical framework specific to medical education is lacking. Although the extant literature has largely focused on determining whether implementation of this design technique is helpful, it is important to explore potential explanatory mechanisms to further understand how and under what conditions gamification is likely to be effective within the context of medical education. We wrote this article to probe both theoretical and practical considerations of applying gamification to medical education. We begin by seeking to define gamification in the context of medical education. Next, we explore how gamification works by applying self-determination theory to gamification principles as a way to explain gamification’s ability to enhance engagement and motivation. Finally, we delineate common roadblocks and challenges of gamification.

Understanding Gamification in the Context of Medical Education

To understand what gamification is, it is important to understand what it is not. Gamification is not a game or a serious game, although these terms have been used interchangeably in the literature. Games are a type of complex, well-designed, structured, rule-based play that are entertaining. Games are competitive with the primary goal of winning.1 Serious games are full-fledged digital games that are designed for a primary purpose other than mere entertainment, such as learning.16 An example of a serious game is LeapPad, an educational game for children. Gamification differs from a serious game in that gamification is a design technique that is applied to an existing learning activity or curriculum to facilitate achievement of the activity’s or curriculum’s goals, whereas serious games begin as a game designed to fulfill objectives specific to the serious game itself. Deploying an actual game within a classroom is not gamification.

Although both gamification and serious games borrow similar concepts of motivation and engagement among participants, they have distinctly different design and implementation concepts.11,17 In particular, gamification curates select game design elements, whereas the serious game acts as a whole entity, often replacing an entire learning activity or course.1 Examples of game design elements commonly applied in medical education are listed in Table 1, the most frequent being points, badges, and leaderboards, collectively referred to as PBL.10 Although gamification examples in medical education are overwhelmingly digital, game design elements can be implemented without digital technology.1,18 Another term that is often confused with gamification is game theory. Although the terms are somewhat similar, gamification and game theory are unrelated. Game theory is a probability-directed framework that predicts decision making and human behavior, such as economic decisions.19 In this article, we focus on gamification in medical education and do not discuss serious games or game theory.

The boundaries between gamification and game and serious game, as well as gamification and simple feedback, are admittedly blurry.1 Adding significant numbers of game design elements to foster learning objectives means the learner is simply playing a serious game. Conversely, a sparse-point system for multiple-choice case vignette questions is not necessarily classified as gamification. There is currently no consensus on how many or what types of game design elements officially constitute gamification, nor is there yet consensus on the nomenclature related to gamification.8 We propose that gamification begins when a purposeful game design element is introduced to facilitate learner engagement, motivation, and behavior changes.

How Gamification Works: Self-Determination Theory as an Explanatory Mechanism

Self-determination theory, a prominent theory of human motivation, provides insights into the motivational effects underlying both successful and unsuccessful gamification and the resulting behavioral outcomes. Self-determination theory posits that learner motivation spans a spectrum from amotivation (i.e., no intention to perform a particular behavior) to extrinsic motivation to intrinsic (internal) motivation.20–23 Each of these motivation states has a regulation—essentially how the learner’s sense of motivation is achieved. Below, we use a medical student in a clerkship to present illustrative examples of the three forms of learner motivation and how they are regulated.

In amotivation, there is no motivation. A student with amotivation has no interest at all, and engagement in the learning activity is nil.

In extrinsic motivation, the regulation of the motivation is either external, introjected, or identified.

  • In external regulation, the learner responds to a concrete reward or punishment, but cannot appreciate the value of the learning activity or learning outcomes aside from that reward or punishment. The medical student completes clerkship exercises for a gift card or to avoid expulsion from school, but has no concern about the actual topic or learning.
  • In introjected regulation, the learner begins to internalize the value or regulatory process, but refuses to build a personal identity from this. The medical student is slightly more motivated and sees the value in completing the clerkship exercises but only because his or her friends are all in the same study session or because the student wants to get a better score than a rival does.
  • Identified regulation refers to internalization of the external regulation, which has finally become part of the learner’s core self and identity. The medical student is motivated to complete the learning exercises because doing that will likely improve his or her standardized test scores in the future for residency. The hope for getting into a competitive residency program is aligned with gaining improved knowledge, and admittance into a residency program of the student’s choice is important enough for the student to internalize the regulation as part of his or her identity (e.g., being a future resident).

In intrinsic motivation, the regulation is intrinsic. The intrinsically regulated student has already completed the exercise without prompting, because this student finds joy and fulfillment in the exercises and learning activities themselves.

Within self-determination theory, intrinsic motivation is more valuable than extrinsic motivation, as the former generally outperforms the latter, leading to more efficient, longer, or more effective learning.8 Self-determination theory posits that sufficient external pressures can allow a learner to internalize the external source of motivation, by fulfilling three basic psychosocial needs: the needs for competence, autonomy, and relatedness.8,20–22 Gamification, as a mediating force to allow learners to internalize motivation, must facilitate these three psychosocial needs.24 Successful gamification does this well, and learners become more motivated to learn because of the gamified system. Poorly constructed gamification will lead to the opposite result, even toward amotivation.

Three key gamification principles outlined in the literature include goal setting, learner control, and engagement.21,25,26 While these concepts are complex and interrelated, they are the tools with which the gamified system can ultimately fulfill the psychological needs of self-determination theory. In the section that follows, we outline how these three psychological needs within self-determination theory may be supported by the parallel concepts of gamification principles. Ultimately, gamification that can meet these needs would, under the self-determination theory framework, move a learner toward intrinsic motivation. Once that level of motivation has been reached, the learner no longer needs gamification—successful gamification “works itself out of a job.”

Three Psychological Needs of Self-Determination Theory and How Gamification Can Facilitate Them

Competence

A learner’s competence refers to the learner’s perception that he or she can successfully achieve a goal.8,20 Goal setting, therefore, is an effective gamification principle that can be used to meet the learner’s need to feel competent. Goal setting theory postulates that there is a positive, linear relationship between specific, challenging goals and performance, with more challenging goals leading to higher performance.24,27

Goals are theorized to improve performance through three specific mechanisms that can be incorporated into educational activities using game design elements. First, difficult goals encourage higher expectations, which in turn increase performance.24 Second, goals from outside sources enhance self-efficacy, or one’s belief that he or she is capable of accomplishing a task. Third, completion of a goal leads to a sense of competence, leading to higher satisfaction and spurring intrinsic motivation.28,29 These effects of goal setting have been demonstrated in varied settings at both the individual and organizational levels.28–30 Goals can reflect activities that promote learning satisfaction and behavior such as repeated practice, or learning outcomes such as higher assessment scores.

To optimally motivate learners, tasks should be created to pose a significant challenge while remaining perceived as attainable.8 Both gamification and medical education experts propose a process of abstracting or deconstructing larger goals and objectives into smaller, discrete components.2,22 This yields not only the overall outcome goal (e.g., successful endotracheal intubation on a simulator mannequin) but also smaller, associated process goals (e.g., laryngoscopy, identification of vocal cords, and passing the endotracheal tube in a simulator mannequin). Process goals can also be related to practice without a specific outcome (e.g., daily simulated intubation practice). Having a variety of attainable goals of varying difficulty should fulfill the need for competence, as greater self-efficacy in novice learners results from using process goals as milestones and opportunities for feedback and practice.22

Game design elements that increase the visibility of attained goals, either to the learner or to others, can also provide feedback on competence. For example, points can be used to quantify different goals, and badges serve as visual symbols of achievement, supporting the competence component of self-determination theory. Leaderboards permit social comparison and a means to display competence to one’s peers. For example, Mokadam et al15 used a ranked leaderboard to increase residents’ use of a small-vessel anastomosis simulator. The addition of this game design element provided a prespecified goal of “winning,” resulting in increased simulator use and a resulting improvement in technical skills.

With regard to CPR skills, MacKinnon et al31 implemented a leaderboard that ranked multiple granular scores separately: frequency of practice, chest compression rate, compression depth, and compression release. Separating smaller component objectives from the overall goal of “high-quality CPR” provided learners with multiple process goals. In other words, leaderboard rankings separated for each process goal was a gamification strategy to fulfill the sense of competence among a greater number of participants. Additionally, it provided learners with more frequent, targeted feedback to inform subsequent performance.

Autonomy

Autonomy refers to the ability of learners to make choices about how they learn, and to providing learners with opportunities to take responsibility for their own learning.23 Games, by definition, are voluntary.32 Part of an activity’s allure is its volitional nature, and the choice to participate in and of itself provides a degree of learner control that enhances both engagement and a sense of autonomy. Also, van Roy and Zaman8 recommend that gamification should provide learners as much control as possible by allowing them the choice of whether to participate or not. An individual’s control over his or her experience is thought to be a crucial component of active learning and is key to the concepts of self-directed learning and self-determination theory.21,30

Learner control also relates to the choices learners can make within the gamified system. If the gamification approach has a large variety of achievements and badges—and consequently has multiple process goals to demonstrate competence—learners can prioritize and choose which goals are most relevant to them.33 Learner control also allows learners to discover weaknesses and strengths that can be useful as they reflect on their learning.

Unique to digital gamification systems, the modality and ease through which the learner interacts with the gamified system can affect the learner’s sense of autonomy. Surgical residents who used Twitter in a gamified microblogging platform were able to access in-service training examination questions despite being in a decentralized residency.34 A paper-and-pencil or even a website format would not have worked. Using optimal gamification design enabled residents to take ownership of their participation. In another example, anesthesia residents who reported procedures using a gamified smartphone app had improved reporting compliance compared with those using a slower, less accessible media platform.35

Relatedness

Relatedness refers to the interconnectedness of the learner to other learners or teachers who facilitate feedback, discussion, and inquiry during the learning experience.23 Appropriate facilitation of the learning experience includes providing a supportive environment and psychological safety for learning.22,23 A safe learning environment offers opportunities for inquiry, reflection, and feedback-seeking behavior.22

Gamification can build relatedness by harnessing the principle of engagement. First, well-designed game design elements provide a choice for learners to either collaborate or compete among their peers.2 Engagement is critical to continued participation in activities.36

In addition, peer-to-peer comparison can stimulate learning and motivation, particularly for individuals who are oriented toward social comparison.37,38 Gamification that uses social media platforms can further add to relatedness, though potentially at a cost to a safe learning environment, because social media can make learners more exposed and vulnerable.

There is empiric research on performance and the sense of relatedness in team-style competitions. In the medical education literature, team-based competition has shown positive outcomes with regard to increased engagement and number of questions answered during online education competitions.14,39 Team-based competition also creates collaborative “learning communities.” In turn, community approval and social influence can act to boost motivation and encourage continued involvement, thereby enhancing learning.40 Internalizing motivation is easier when shared among family, friends, and other trusted individuals.8

Relatedness can also be influenced by the user interface, just like autonomy. MacKinnon et al31 conducted a successful paper-and-marker leaderboard for CPR skills competition among health care providers at a single institution. To increase the breadth of relatedness to a multicenter competition, a new technologically based user interface was required. Chang et al41 describe the development of a digitally based leaderboard using gamification principles to engage relatedness, including the use of an online leaderboard that provides updates in real time and ability to share photographs and selfies amongst participants. These smaller game design elements encourage engagement and fulfill the need for relatedness.

Challenges in Implementing Gamification in Medical Education

When learners fulfill the three psychological needs—a sense of autonomy, a perception of competence, and relatedness—self-determination theory states that they are more apt to internalize motivation. However, not all gamification approaches succeed in addressing these needs, and not all empirical studies of gamification in education show improved outcomes.1,8 For example, Hakulinen et al42 found that badges had no effect on assessment scores despite improved learning behavior, and Van Nuland and colleagues’43 research showed that an online anatomy knowledge tournament had minimal clinical effect. Although gamification is increasingly being applied in the medical education context, there is still insufficient evidence to support a claim of long-term benefits of this design technique.10 We propose that some of the failures in gamification can be explained in line with self-determination theory, and elaborate below on two explanations: the overjustification effect and negative effects of competition.

Overjustification effect

The overjustification effect is the net negative effect on engagement and motivation from an overreliance on external motivating regulations.8,44 That is, adding game design elements to increase extrinsic motivation can have an adverse impact on learners who already started with a strong intrinsic motivation.45 Overreliance on external sources of motivation tends to remove any learner control and thereby adversely affects autonomy.46 Oliver and Williams46 maintain that a gamified system trivializes learners’ intrinsic motivation and sense of competence, especially for advanced learners such as medical students and residents. Empirical studies45,47 examining whether a loss of intrinsic motivation occurs with the addition of extrinsic rewards do not show consistent results. Research to date indicates that the longitudinal effect of rewards is highly dependent on the baseline interest of participants. If baseline interest is high—meaning the activity itself captures and engages participants—then adding extra rewards leads to overjustification and loss of intrinsic motivation.20,21 Essentially, the goal of the game conflicts with the learning goals.

The choice of game design elements also influences the risk of overjustification in highly motivated learners. Tangible awards (e.g., prizes or monetary awards) can be insulting or even viewed as bribery in the context of medical education and are clearly externalized goals. It is particularly difficult to align the motivation to earn instant cash with the intrinsically regulated motivation to find pleasure or meaning in the learning activity itself; it reflects a schism in the concept of autonomy within self-determination theory. Additionally, a learner’s perception of competence can break once an extrinsic reward is placed within a gamified system. One example of this extrinsic reward in gamification is the badge. When internal medicine residents were asked about their leaderboard competition online, Nevin et al14 found that badges were perceived as the least motivating element within their study. Other empirical studies7,42,48,49 examining the effect of badges demonstrate a motivational effect in a very narrow target audience: participants who are relatively skilled but are otherwise poorly motivated. Badges per se improve only amotivated learners within the context of self-determination theory.45 Learners who are otherwise beginning to internalize their motivations can be undermined by externalized rewards or punishments.

Poorly designed gamification that invokes the overjustification effect often stems from a frustration over poor learner engagement, when, in actuality, the poor learner engagement may be related to the underlying instructional design. Gamification can only mediate and augment appropriate instructional design; it cannot replace or supplant it.11 Steps to avoid overjustification include starting with a strong instructional design and not relying on gamification to salvage poor instruction. Second, simply applying a game design element such as badges, without contextualizing it to meaningful goals, can lead to an overjustification effect.8 A gamified experience that fully engages the user through carefully constructed game design elements, taking into account both the activity being gamified and the target participants, is much more likely to succeed.3,7 Consideration of the potential for either negative or positive effects on motivation is key in choosing which systems to gamify, which game design elements to use, and which participants are most likely to benefit.

Negative effects of competition

Self-determination theory purports that the stress of competition or the stress from inferior performance leads to a poorer sense of competence and even autonomy. When the competition is synchronous or public and the lack of achievement is prominently displayed, relatedness—particularly to learner peers—can also suffer. Landers and Landers50 describe this well: “the relationship between conflict and effort is curvilinear, with an ideal level of conflict at a middle ground.”50 Either frustrated or unchallenged learners will demonstrate poor engagement and poor time-on-task, leading to inferior learning outcomes. The optimal leaderboard would ensure roughly equal chances of achieving a specified rank on the leaderboard, given equal effort.50 However, individuals who rank low may still find their intrinsic motivation hindered by the visibility afforded by the leaderboard.

Competition can also be fatiguing, and longevity of competition—such as that invoked through leaderboards—may be difficult. Nevin et al14 found a 33% initial attrition rate (analogous to amotivation) amongst residents competing in a medical knowledge competition, and MacKinnon et al31 found a 27% attrition rate for simulated infant CPR practice. It is likely that relatedness suffers when leaderboard rankings change minimally over time. Theoretically, learners who feel that they are unable to ascend the leaderboard should perceive a lack of competence, driving them toward amotivation.8,22 This is supported by empirical data on self-reported demotivation among participants whose performances were visibly poor on a leaderboard.14,41

Steps to minimize negative effects of competition include maximizing any collaborative opportunities (e.g., team-based competition). Although there is evidence to support the use of game design elements that spur competition,24 too much competition can undermine relatedness and competence. A safe learning environment is also recommended, and anonymizing, or deidentifying performance data, may help.

Summing Up

Gamification has become a popular addition to medical education curricula in an attempt to engage the adult learner. Whereas past gamification research has largely focused on determining whether implementation of gamification leads to better learning outcomes in the medical education context, current research priorities in gamification should explore how and under what conditions gamification is likely to be effective. Selective and purposeful gamification that aligns with learning goals has the potential to increase learner motivation and engagement and, ultimately, learning. In line with self-determination theory, game design elements can be used to enhance learners’ feelings of relatedness, autonomy, and competence to foster learners’ intrinsic motivation. Poorly applied game design elements, however, may undermine these basic psychological needs by the overjustification effect or through negative effects of competition. Educators must, therefore, have a clear understanding of the benefits and pitfalls of gamification in curricular design, take a thoughtful approach when integrating game design elements, and consider the types of learners and overarching learning objectives.

Acknowledgments: The authors would like to thank the Quality Cardiopulmonary Resuscitation (QCPR) leaderboard investigators of the International Network for Simulation-based Pediatric Innovation, Research, and Education (INSPIRE) for their guidance in the preparation of this manuscript. A list of the QCPR leaderboard investigators may be found in Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/A531.

References

1. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: Defining gamification. In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. 2011:915. https://dl.acm.org/citation.cfm?id=2181037. Accessed January 14, 2018.
2. Yunyongying P. Gamification: Implications for curricular design. J Grad Med Educ. 2014;6:410412.
3. Kapp KM. The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. 2012.San Francisco, CA: Pfeiffer.
4. Dale S. Gamification: Making work fun, or making fun of work? Bus Inf Rev. 2014;31:8290.
5. Jakubowski M. Gamification in business and education—Project of gamified course for university students. Dev Bus Simul Exp Learn. 2014;41:339342.
6. Baranowski MT, Lieberman PD, Buday R, et al. Videogame mechanics in games for health. Games Health J. 2013;2:194204.
7. Hamari J, Koivisto J, Sarsa H. Does gamification work? A literature review of empirical studies on gamification. In: Proceedings of the 2014 47th Hawaii International Conference on System Sciences. 2014:30253034. https://dl.acm.org/citation.cfm?id=2584909&picked=prox. Accessed January 14, 2018.
8. van Roy R, Zaman B. Why gamification fails in education—And how to make it successful. Introducing 9 gamification heuristics based on self-determination theory. In: Serious Games and Edutainment Applications. 2017:Vol II. Cham, Switzerland: Springer; 485509.
9. Cooper S, Khatib F, Treuille A, et al. Predicting protein structures with a multiplayer online game. Nature. 2010;466:756760.
10. Dichev C, Dicheva D. Gamifying education: What is known, what is believed and what remains uncertain: A critical review. Int J Educ Technol Higher Educ. 2017;14:9.
11. Landers RN. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simul Gaming. 2014;45:752768.
12. Chen PH, Roth H, Galperin-Aizenberg M, Ruutiainen AT, Gefter W, Cook TS. Improving abnormality detection on chest radiography using game-like reinforcement mechanics. Acad Radiol. 2017;24:14281435.
13. Kerfoot BP, Kissane N. The use of gamification to boost residents’ engagement in simulation training. JAMA Surg. 2014;149:12081209.
14. Nevin CR, Westfall AO, Rodriguez JM, et al. Gamification as a tool for enhancing graduate medical education. Postgrad Med J. 2014;90:685693.
15. Mokadam NA, Lee R, Vaporciyan AA, et al. Gamification in thoracic surgical education: Using competition to fuel performance. J Thorac Cardiovasc Surg. 2015;150:10521058.
16. Graafland M, Schraagen JM, Schijven MP. Systematic review of serious games for medical education and surgical skills training. Br J Surg. 2012;99:13221330.
17. Garris R, Ahlers R, Driskell J. Games, motivation, and learning: A research and practice model. Simul Gaming. 2002;33:441467.
18. Chang TP, Kwan KY, Liberman D, et al. Introducing teamwork challenges in simulation using game cards. Simul Healthc. 2015;10:223226.
19. Blake A, Carroll BT. Game theory and strategy in medical training. Med Educ. 2016;50:10941106.
20. Deci EL, Ryan RM. Self-determination theory: A macrotheory of human motivation, development, and health. Can Psychol. 2008;49:182185.
21. Cook DA, Artino AR Jr. Motivation to learn: An overview of contemporary theories. Med Educ. 2016;50:9971014.
22. Schumacher DJ, Englander R, Carraccio C. Developing the master learner: Applying learning theory to the learner, the teacher, and the learning environment. Acad Med. 2013;88:16351645.
23. Orsini C, Evans P, Jerez O. How to encourage intrinsic motivation in the clinical teaching environment? A systematic review from the self-determination theory. J Educ Eval Health Prof. 2015;8. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4397857/. Accessed January 14, 2018.
24. Landers RN, Bauer KN, Callan RC, Armstrong MB. Psychological theory and the gamification of learning. In: Gamification in Education and Business. 2015:Cham, Switzerland: Springer; 165179.
25. Bedwell WL, Pavlas D, Heyne K, Lazzara EH, Salas E. Toward a taxonomy linking game attributes to learning: An empirical study. Simul Gaming. 2012;43:729760.
26. Landers RN, Bauer KN, Callan RC. Gamification of task performance with leaderboards: A goal setting experiment. Comput Human Behav. 2017;71:508515.
27. Latham GP, Locke EA. New developments and directions for goal-setting research. Eur Psychol. 2007;12:290300.
28. Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation. A 35-year odyssey. Am Psychol. 2002;57:705717.
29. Weldon E, Jehn KA, Pradhan P. Processes that mediate the relationship between a group goal and improved group performance. J Pers Soc Psychol. 1991;61:555569.
30. Brydges R, Mallette C, Pollex H, Carnahan H, Dubrowski A. Evaluating the influence of goal setting on intravenous catheterization skill acquisition and transfer in a hybrid simulation training context. Simul Healthc. 2012;7:236242.
31. MacKinnon RJ, Stoeter R, Doherty C, et al. Self-motivated learning with gamification improves infant CPR performance. A randomised controlled trial. BMJ Simul Technol Enhanc Learn. 2015;1:7176.
32. McGonigal J. Reality Is Broken: Why Games Make Us Better and How They Can Change the World. 2011.New York, NY: Penguin.
33. Orvis KA, Fisher SL, Wasserman ME. Power to the people: Using learner control to improve trainee reactions and learning in web-based instructional environments. J Appl Psychol. 2009;94:960971.
34. Lamb LC, DiFiori MM, Jayaraman V, Shames BD, Feeney JM. Gamified Twitter microblogging to support resident preparation for the American Board of Surgery in-service training examination. J Surg Educ. 2017;74:986991.
35. Avidan A, Weissman C, Levin PD. Integration of QR codes into an anesthesia information management system for resident case log management. Int J Med Inform. 2015;84:271276.
36. Bouvier P, Lavoué E, Sehaba K. Defining engagement and characterizing engaged-behaviors in digital gaming. Simul Gaming. 2014;45:491507.
37. Raat J, Kuks J, Cohen-Schotanus J. Learning in clinical practice: Stimulating and discouraging response to social comparison. Med Teach. 2010;32:899904.
38. Marczewski A. Marczewski A. User types. In: Even Ninja Monkeys Like to Play: Gamification, Game Thinking and Motivational Design. 2015.Addlestone, UK: Gamified UK.
39. Scales CD Jr, Moin T, Fink A, et al. A randomized, controlled trial of team-based competition to increase learner participation in quality-improvement education. Int J Qual Health Care. 2016;28:227232.
40. Mann KV. Theoretical perspectives in medical education: Past experience and future possibilities. Med Educ. 2011;45:6068.
41. Chang TP, Doughty CB, Mitchell D, et al.; INSPIRE In-Hospital QCPR Leaderboard Investigators. Leveraging quick response code technology to facilitate simulation-based leaderboard competition. Simul Healthc. 2018;13:6471.
42. Hakulinen L, Auvinen T, Korhonen A. Empirical study on the effect of achievement badges in TRAKLA2 online learning environment. In: Learning and Teaching in Computing and Engineering (LaTiCE), 2013. IEEE Explore Digital Library:4757. http://ieeexplore.ieee.org/document/6542238/. Accessed March 22, 2018.
43. Van Nuland SE, Roach VA, Wilson TD, Belliveau DJ. Head to head: The role of academic competition in undergraduate anatomical education. Anat Sci Educ. 2015;8:404412.
44. deMarcos L, Dominguez A, Saenz-de-Navarrete J, Pages C. An empirical study comparing gamification and social networking on e-learning. Comput Educ. 2014;75:8291.
45. Cameron J, Banko KM, Pierce WD. Pervasive negative effects of rewards on intrinsic motivation: The myth continues. Behav Anal. 2001;24:144.
46. Oliver R, Williams R. Performance patterns of high, medium and low performers during and following a reward versus non-reward contingency phase. Sch Psychol Q. 2006;21:119147.
47. Cameron J, Pierce WD, Banko KM. Achievement-based rewards and intrinsic motivation: A test of cognitive mediators. J Educ Psychol. 2005;97:641655.
48. Domínguez A, Saenz-de-Navarrete J, De-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ. Gamifying learning experiences: Practical implications and outcomes. Comput Educ. 2013;63:380392.
49. Denny P. The effect of virtual achievements on student engagement. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM Digital Library. 2013:763772. https://dl.acm.org/citation.cfm?id=2470763. Accessed March 22, 2018.
50. Landers RN, Landers AK. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simul Gaming. 2014;45:769785.

References cited in Table 1 only

51. Seehusen DA, Ledford CJ, Grogan S, et al. A point system as catalyst to increase resident scholarship: An MPCRN study. Fam Med. 2017;49:222224.
    52. Doherty I, Sharma N. Aligning the use of portfolios with digital badging. Br J Hosp Med (Lond). 2015;76:596598.
      53. Scaffidi M, Pearl M, Walsh CM, et al. Gamification of a virtual-reality simulation curriculum in endoscopy: A randomized controlled trial. Cureus. www.cureus.com/posters/1306-gamification-of-a-virtual-reality-simulation-curriculum-in-endoscopy-a-randomized-controlled-trial. Published November 30, 2017. Accessed February 11, 2018.

        Supplemental Digital Content

        Copyright © 2018 by the Association of American Medical Colleges