Competency-based medical education begins with a careful consideration of competencies required by medical graduates to address health care priorities and provides a mechanism to integrate the health needs of the country with the values of the medical profession.54,57 Competencies describe the complex set of knowledge, skills, and attitudes a medical student requires upon graduation.21,72 Articulation of these competencies is guided by the expectations and requirements of the health care system, including professional organizations, medical colleges, government, and, importantly, consumers. Competency-based education focuses on defining learning outcomes that can be measured to determine mastery of a specified competency upon graduation.54 These learning outcomes influence the core educational activities of teaching, learning, and assessment.57
Robust assessment and documentation of student expertise is integral for the effective implementation of competency-based medical education.74 Assessment in medical education can be defined as the processes used to make judgements about the achievements of students over a course of study.59 Important changes have been made to methods of assessment of medical students over the last 5 decades.102 Historically, assessments were designed to determine students' performance through an end of year examination. However, many medical schools now undertake continuous assessments of students' mastery of educational objectives.10 A number of standardised methods and instruments have been developed to provide accurate, reliable, and timely assessments of medical students' competencies across a comprehensive curriculum.32,102
Pain medicine is the medical discipline that aims to prepare students to care for patients living with pain.8 It deals with the science, prevention, evaluation, treatment, and rehabilitation of persons living with pain from a multimodal perspective. Pain medicine is a developing field of medical practice that recently gained recognition as a medical speciality in a number of countries such as Australia, Canada, New Zealand, United Kingdom, and United States.36,83,98,116 Acute pain is one of the most common reasons for patients to seek treatment at an emergency department, and severe pain after surgery remains a major problem.51 Chronic pain affects approximately 20% of the population and is a significant public health problem with important biomedical, psychological, social, and economic implications.17,27,62,65,78,100 It is critical, therefore, that medical students are equipped with competencies in the field of pain medicine, and that these competencies are adequately assessed, so that the students are prepared for the clinical environment upon graduation. However, pain medicine education is poorly addressed in medical school curricula internationally.104
Pain medicine competencies may be defined as the observable abilities of medical students (and practitioners) to integrate knowledge, skills, and attitudes related to pain medicine, into effective clinical practice.41,43 Traditionally, pain medicine for new medical graduates was conceptualised as a knowledge-based subject rather than a set of learned competencies in which students must demonstrate proficiency.13 The International Association for the Study of Pain (IASP) is the leading global professional forum for science, practice, and education in the field of pain.66 Pain medicine curricula for medical undergraduates were first developed by the IASP in 1988,91 and a number of editions have been published over the intervening years, the most recent in 201823,67 (appendix 1, available at http://links.lww.com/PR9/A36). The objective of the IASP Curriculum Outline on Pain for Medicine is to provide the knowledge and skills necessary for new graduates to advance the science and management of pain as part of an interprofessional team.67 Desired outcomes of pain medicine education emphasize the learner's capacity to successfully and compassionately perform tasks in the real world, such as pain assessment, collaborative approaches to treatment options, and application of pain competencies across the lifespan in the context of various settings, populations, and care-team models.67 More recently, core competencies for pain management with measurable learning outcomes were developed by an Expert Interprofessional Pain Competencies Consensus Group to provide guidance related to prelicensure health professional education (Fishman et al.)41 (appendix 2, available at http://links.lww.com/PR9/A36). Internationally, there is poor documentation of pain medicine competency assessment methods for medical students.19,81,104 Questions about how best to assess these competencies remain unanswered.13
Consequently, the aim of this review was to identify and summarise the literature regarding methods for assessing pain medicine competencies in medical students. In particular, the following questions were addressed:
- What was the purpose of the assessment? (eg, was it to evaluate the effectiveness of a course or to evaluate students' performance in an examination?)
- How were the assessment instruments developed? (eg, were they developed with attention to specific learning objectives/competencies; with reference to the IASP curriculum and/or interprofessional learning?)
- What level of clinical competency was assessed? (eg, knows, knows how, shows how, and does82)
- What type of assessment methods was used? (eg, multiple choice questions [MCQs], Objective Structured Clinical Examination [OSCE])
- What topics of pain medicine education were assessed?
- What attention was paid to the reliability and validity of the assessment tools?
- What learning domains were assessed? (eg, knowledge, skills, and/or attitudes)
These data will be used to develop a framework for the rigorous assessment of pain medicine competencies for medical students.
2.1. Literature search
Pubmed, Medline, Embase, ERIC, Google Scholar, and BEME were searched for articles relating to the assessment of medical students' competencies related to pain medicine. The following search terms were used: pain, student, undergraduate, medical, education, knowledge, attitudes, skills, and curriculum. The term “pain” was included in all the searches (appendix 3 for the Medline search strategy, available at http://links.lww.com/PR9/A36). Secondary hand-searching of reference lists and relevant publications known to the researcher identified 3 further studies, which were also included.
2.2. Screening and inclusion/exclusion criteria
Articles were selected by reviewing titles and abstracts to determine whether they might meet the inclusion criteria. Two reviewers independently screened all identified articles after deduplication. The full text of the article was retrieved where relevance was unclear. Consensus was sought through discussion where the decision regarding eligibility of an article was unclear. The article was not included if both reviewers agreed the article was not relevant to the research question.
The main inclusion criteria for this review were as follows:
- Medical students (defined as undergraduate or graduate-entry students undertaking a course of study at a medical school leading to the primary qualification of medical practitioner);
- Pain medicine (defined as the discipline within the field of medicine that is concerned with the science, prevention, evaluation, treatment, and rehabilitation of persons living with pain);
- Empirical studies published between January 1, 1997, and December 31, 2016, describing assessment of any domain of pain medicine competencies (knowledge, attitudes, or skills) in medical students.
Exclusion criteria included:
- Studies that were not focussed primarily on assessment of knowledge, attitudes, or skills related to pain medicine;
- Articles that focussed solely on palliative care education without referencing pain medicine for medical students;
- Studies that investigated knowledge, attitudes, and skills in pain medicine amongst groups other than medical students;
- Continuing medical education and medical courses offered to practitioners who have graduated from their medical degree; and
- Editorials and review articles.
2.3. Data extraction
All articles selected were analysed according to the following headings: Details of study (name of first author; date of publication; university/institution; country of study; study population; and course); study purpose; assessment instrument development (eg, relationship to education course content, published literature, and IASP curriculum); level of assessment; method of assessment; pain medicine topics assessed; psychometric properties of the instrument (eg, validity and reliability); and domains of learning measured.
Eight hundred thirty-four citations were retrieved from the initial database search after removal of duplicates. A total of 41 published studies described 53 assessment instruments that were used to examine pain medicine competencies of medical students. The details of the selection process are shown in Figure 1.
3.1. Characteristics of studies
Studies included 7599 medical students across the different years of study, from first to final year. The number of participants in a study ranged from 1246 to 1181,63 with a median of 114 participants.
Just over half of the studies (51%) were published before 2010 and 98% were in English. One study was published in German. Twenty-two studies (54%) were from North America (United States 44%9,24,25,34,55,60,71,80,85,92,99,107–110,127,128,132; Canada 10%63,90,114,125); 12 (29%) from Europe3,6,12,46,56,69,73,84,87,94,95,101; 3 (7%) from Australia;18,118,120 and 1 each from Saudi Arabia,68 Philippines,76 Thailand,88 and Taiwan.50
3.2. Purpose of pain medicine assessments
The purpose of the assessments as described in the articles reviewed is outlined in Table 1. More than half of the studies (56%) assessed medical students who were exposed to a specific pain medicine module within the medical curriculum, such as a comprehensive pain medicine course (20%),12,34,46,63,85,110,125,132 cancer pain module (15%),50,71,76,88,92,108 paediatric pain course (5%),9,24 acute pain module (5%),55,99 or low-back pain module (5%).56,127 Four studies (10%) described an instrument that was used for assessment of interprofessional pain education.34,63,99,125
3.3. Resources used to develop assessment instruments
Information on the resources used to design assessment tools were provided in 28 (68%) studies (Table 2). Most studies based test content on literature review (34%) or consultation with faculty experts (29%) rather than on defined learning objectives or competencies.
Four studies referenced published schemes that describe educational competencies in more detail.9,34,90,132 Only one study cited the core competencies for pain management for prelicensure health professionals published in 2013 by Fishman et al.41 Two other schemes were referenced: namely, the Accreditation Council for Graduate Medical Education1 core competencies and the Educating Future Physicians in Palliative and End-of-Life Care2 core competencies both of which were adapted for use in a pain medicine setting.
3.4. Assessment of levels of clinical competence
Using Miller's pyramid of clinical competence, assessment methods can be classified in terms of what level of competence they require of the student such as knows, knows how, shows how, and does.82 Most studies (80%) in this review assessed “knows”—straight factual recall of knowledge using MCQs and short answer tests, and “knows how”—the application of knowledge to problem-solving and decision-making using written assignments, oral questioning, and development of management plans. Eight (20%) studies focused on “shows how,”55,56,73,101,108–110,127 examining integration of learning skills into a successful performance with a standardised patient (SP) or simulated experience. No studies examined students' actual performance in a real-life clinical setting on the wards or in the consulting room.
3.5. Standard setting
Standard setting is the process adopted to define levels of proficiency, typically but not limited to agreement on a defensible pass score.86 Although standard setting is generally accepted as good practice in medical education, only 5 studies (12%) gave details of a defined standard of competence (eg, pass/fail) to evaluate the students' knowledge, attitude, or clinical skills.63,76,85,110,127
3.6. Methods used to assess pain competencies
Table 3 provides information on the methods used in the studies to assess pain medicine competencies. Thirteen studies (32%) used more than one instrument or method of assessment, with 11 studies using 2 instruments9,56,60,63,69,71,95,99,101,114,128 and 2 studies using 3 instruments.18,85
Nineteen studies used existing instruments or modified existing instruments for assessment of medical students' knowledge, attitudes, or skills. The Pain Knowledge and Beliefs Questionnaire,63,125 the Cancer Pain Objective Structured Clinical Examination,108,109 the Back Pain Beliefs,18,69,112 Health Care Providers' Pain and Impairment Relationship Scale,18,84,96 the Cancer Pain Questionnaire,50,68,129 and Paediatric Pain Knowledge Questionnaire9,24 were each used in 2 studies. Most of these existing instruments were developed before 2005 (n = 17 89%). New instruments were designed in 78% (n = 32) of studies.6,9,12,24,25,34,46,55,56,60,63,68,71,73,80,85,87,88,90,92,94,95,99,101,109,110,114,118,120,125,127,132
3.6.1. Written examination
Written examinations were used in 31 studies to assess students' pain medicine competencies. The number of questions per test ranged from 1 to 59, with a mode of 13. Three studies did not report the number of questions in the assessment tool.
184.108.40.206. Multiple choice questions and Likert rating scales
Multiple choice questions were used to assess knowledge in 17/18 studies (94%). Single best answer format of the MCQ was used in 11 studies6,9,12,24,68,85,88,90,94,95,101 and 10 studies used the true/false format.3,12,50,63,68,76,90,118,125,132 Likert rating scales were used to measure pain-related knowledge and attitudes, and the number of response options in the rating scales ranged from 3 to 7. There was limited use of open-ended short answer questions and structured essays.
220.127.116.11. Group assignments
A small-group assignment, which involved completion of a comprehensive pain management care plan, was evaluated as an indication of transfer of learning to the real-life patient context.63 One study used a paired-work assignment to assess clinical applications of cancer pain.85
18.104.22.168. Images, vignettes, or written description of a clinical scenario
Still facial images (1 study), vignettes (3 studies), and written description of a clinical scenario (2 studies) were used to examine medical students' treatment recommendations and attitudes towards patients with pain,18,60,94,101,120 in particular, the influence of contextual variables on clinical recommendations. A “key features” problem was used in one study, which involved the use of brief clinical scenarios of patients in pain, to evaluate the students' pain diagnostic and management skills.101 In these 5 studies, students recorded answers using a 0 to 100 visual analogue scale, by choosing answers from a list of possible choices or using a Likert scale. The number of clinical scenarios ranged from 1 to 16 with a median of 8.
22.214.171.124. Reflective journals
Reflective journals were examined in one study to explore students' attitudes to working with patients with chronic pain.25 First-year students described their experiences during a 4-week clinical immersion rotation. Entries were considered as students' perceptions of pain encounters if they included expressions of thoughts and feelings engendered by pain-related clinical experiences.25 The journal entries were coded and recoded using an iterative process.25
3.6.2. Clinical simulation
126.96.36.199. Objective Structured Clinical Examination
The OSCE was used in 8 studies,55,56,73,101,108–110,127 primarily to assess clinical skills (88%), but also to assess knowledge (37%)55,108,109 and attitude to pain (13%).101 One OSCE focussed primarily on regional anaesthesia techniques,55 2 studies assessed clinical competence in the area of cancer pain management,108,109 and 2 studies assessed clinical skills in examining chronic low-back pain.56,127 The number of OSCE stations ranged from 1 to 10, with a median of 3. One study included a global rating scale for clinical and communication skill checklists.73
188.8.131.52. Simulation-based group assessment
In one study, medical-nursing student dyads were challenged to assess and manage acute pain in a simulated hospitalized patient using effective collaborative interprofessional skills.99 The students completed presurvey and postsurvey measuring confidence in their ability to assess and manage acute pain.
3.6.3. Assessment by supervising clinicians
184.108.40.206. Focus groups
Focus groups were used in 2 studies to explore the knowledge and attitudes of students to pain management, retention of material presented in the course, and to identify gaps in knowledge with respect to pain education.71,114 Students were asked a set of questions on specific topics such as the meaning of pain (opiophobia and subjectivity), assessment of pain, clinical management of pain (particularly with regard to opioids), communication and the therapeutic relationship, ethical considerations, and the institutional context influencing the students' attitudes to providing pain management. Both studies used a semistructured format.
220.127.116.11. Interprofessional workshops
Medical and nursing students collaborated to develop a best-practice pain management plan for 2 case studies.34 A scoring checklist for the interprofessional pain management plans was used to assess and compare learning outcomes between student groups.
18.104.22.168. Group-based oral assessment
One study included an oral examination to assess students' learning.95 The examination took the format of a small-group discussion during which clinical cases were presented and teachers evaluated students' knowledge.
3.6.4. Multisource assessment
Written portfolios were used to assess students' pain attitudes in one study.85 The portfolios involved a compilation of short exercises including writing a brief pain narrative, describing the pain depicted in a fine-art image, assessing personal responses to the experience of pain during a common psychophysical test (the 30-second cold pressor test), drafting personal learning objectives for the pain course, defining the role of empathy and compassion in medicine, a description of the relationship between pain and addiction, and a reflection of lessons learnt from a pain expert panel and positive personal qualities exhibited by the pain expert clinicians.85 The portfolios were graded by one examiner using a 4-point system (fair, good, very good, and excellent).
Six studies described medical students' self-assessment of pain knowledge competencies and confidence in their clinical capabilities.9,46,56,76,92,99 Students rated their competencies with an agreement or disagreement response option towards a given statement, using a 4- or 5-item Likert response format.
3.7. Pain medicine topics included in assessment tools
There was considerable variation in the attention given to different topics (Fig. 2) (appendix 1, available at http://links.lww.com/PR9/A36). The IASP core curriculum for medical students was used to benchmark the pain content topics that were evident in the reviewed studies.67
Clinical pharmacology was the most commonly assessed topic (n = 31; 76%), which included opioids (n = 22; 71%), adjuvant analgesics (n = 4; 13%), local anaesthetics (n = 2; 7%), and nonsteroidal anti-inflammatory drugs (n = 2; 7%). Some studies did not specify details of pharmacology topic assessed.
3.8. Validity in assessment
Details of the psychometric properties of the assessment tools varied. Validity was discussed to some degree in 16 studies (39%),6,18,50,60,63,68,69,71,84,99,108–110,118,125,127 with content validity being most frequently reported (15%).6,50,63,76,118,125 Validity of OSCE assessments were reported as being improved by strategies such as the incorporation of SPs,55,56,73,101,108–110,127 use of a checklist,55,56,73,101,108–110,127 inclusion of 3 or more stations,73,108–110,127 and blinded observers.55,101,127 Fourteen studies (34%) discussed the reliability of assessment instruments. Information on the psychometric properties of instruments used was not provided in 13 studies (32%).9,12,24,46,85,88,90,92,94,95,120,128,132
3.9. Pain medicine learning domains assessed
Studies assessed the following pain medicine domains: knowledge (n = 24; 59%), attitudes of medical students towards patients in pain and to the discipline of pain medicine (n = 23; 56%), and clinical skills necessary for the practice of pain medicine (n = 14; 34%) (Table 4). Most assessment methods evaluated core knowledge, attitudes, and clinical skills in isolation. Comprehensive evaluation of learning encompassing all 3 dimensions of knowledge, attitudes, and skills was described in 3 studies (7%).
This review provides an overview of methods used to assess medical students' competencies in pain medicine internationally. Studies were predominantly focussed on assessment of pain medicine knowledge and attitudes of North American medical students before 2010. Most studies described summative assessments using written tests. Only one-third of studies used assessment tools based on internationally recognised pain medicine curricula/learning objectives (such as the IASP Curriculum Outline on Pain for Medicine)67 or core competencies for pain management (Fishman et al.).41 The most common pain medicine education topics assessed were pain pharmacology (in particular, opioid prescription) and the management of cancer and low-back pain. Validity of the assessment tools was poorly addressed in general. Most studies focussed on assessment of the cognitive level of learning (such as knowledge and understanding of pain medicine) as opposed to the more challenging domains of demonstrating skills and attitudes, and developing and implementing pain medicine management plans.
Worldwide, entry-level medical education programmes are tertiary courses undertaken at a medical school.130 The duration of these courses varies between 4 and 6 years.130 In many countries, newly graduated doctors are specifically required to undertake a period of clinical practice (internship, foundation programme, or housemanship) lasting 1 to 2 years before full registration or unrestricted practice.130 During this bridging period after medical school, new graduates are generally required to practice the key skills and knowledge learnt during their medical education, so as to gain proficiency in the basic clinical skills that will prepare them for the context in which they will be expected to work.48,131 Pain and inadequate pain management remain a significant problem in hospitals.35,51,106,111,122 New graduates are likely to encounter patients experiencing acute, chronic noncancer and cancer pain in hospital emergency, surgical, and medical departments.
A recent review of pain medicine education internationally has revealed that, in general, comprehensive pain medicine content is not mandatory in the medical curriculum.103 Pain medicine is taught in a fragmented way within modules in specific areas of medicine (such as anaesthesia) as opposed to a dedicated pain module.103 Minimal learning time is allocated to the topic of pain medicine.103 In Germany, education on pain became compulsory in 2012 within federally defined medical school curricula.19 The Pain Management Core Curriculum for German Medical Schools has been developed to provide the content for a defined set of teaching sessions in interdisciplinary pain management for medical students.67 This curriculum formed the basis of the recently updated Pain Management Code Curriculum for European Medical Schools.67 The IASP Curriculum Outline on Pain for Medicine is based on these curricula.67
A paucity of research into the assessment of pain medicine competencies indicates a general lack of emphasis on pain medicine assessment internationally.19,81,104 Assessment provides a reflection of institutional values.33 Explicit value can be judged by the amount of time and resources that are allocated to the teaching of a topic and the weighting of the topic in the assessment process. The absence of attention to assessment of pain medicine competencies implies a prevailing lack of value in the topic by teaching institutions.32 The current approach to medicine is based largely on the disease model.105 Historically pain was regarded as a passive symptom of a disease process.105 However, there is accumulating evidence that pain is a disease entity with its own pathology, symptoms, and signs.105 In addition, most medical schools focus on a biomedical model and emphasize knowledge-based learning.75 Although neurobiological and pathology of pain mechanisms are important aspects of the pain curriculum, the psychosocial dimensions of pain development, maintenance, and management are critical to pain medicine.75 This broader conception of pain medicine sets it apart from other fields of medicine such as genetics, radiology, and surgery.
4.1. Purpose of assessment
This review demonstrated that assessments of medical students' pain medicine competencies were undertaken for a limited number of purposes. Most studies were performed from the perspective of assessing efficacy of a pain course or identifying gaps in the medical curriculum. There are a number of diverse personal, institutional, and societal reasons why assessment of pain medicine competencies of medical students should be an integral part of a competency-based medical curriculum (Box 1).33 The goal of the assessment will determine the approach to the assessment process, that is, summative assessments of learning and/or formative assessments for learning. These assessment goals are not necessarily discrete or opposed to one another, as in some instances, they may be interdependent and complimentary.121
Fundamental purposes of assessment of pain medicine competencies of medical students
For the student
Facilitates a desire to learn and develop pain medicine skills
Focuses the learner's attention on what is considered core pain knowledge and competencies
Provides feedback about performance in pain medicine assessments to identify strengths and weaknesses
Encourages students to reflect on their own performance and attitudes to pain medicine
For the teacher
Demonstrates achievement of pain competencies
Identifies educational gaps in the curriculum with respect to pain medicine
Promotes curriculum development
Recognises standards of competence for students at different levels
For the institution
Ensures that the medical workforce is able to practice pain medicine in an interprofessional model of care
Meets the demands of the accrediting and professional bodies with respect to defined graduate outcomes
Reflects the institutional value placed on pain medicine by determining how pain medicine is assessed
Recognises the “hidden” curriculum
Justifies progression within the medical curriculum
For the health system
Determines which practitioners are competent and safe to enter the workforce
Optimises careful use of limited resources
Promotes best-practice pain care for the patients
For the public
Provides accountability in terms of medical professionals' abilities to integrate pain-related knowledge, skills, values, and attitudes into clinical practice
4.1.1. Assessment of learning
Most studies in this review described summative assessments to pilot a new pain course, to improve an existing pain module, or to formulate pain education resources for students. There was a significant lack of high-stakes assessments to determine progression through a course (7%) or achievement of a set standard of competence (12%). The public expects medical schools to certify the competence of future practitioners to meet the health care needs of people experiencing pain.40 There have been calls for professional bodies to increase the number of required entry-to-practice pain competencies to improve the capacity for medical practitioners to alleviate the suffering of unrelieved pain.126 More direction is needed to define criteria or standards for these assessments to support high-stake decisions regarding readiness for entry into unsupervised medical practice and to promote best-practice pain care for patients.
The review revealed limited examples of pain medicine competency assessments based on the defined IASP Curriculum objectives (28%) or core competencies for pain management for prelicensure health professionals (Fishman et al.) (2%). A possible reason for this lack of integration may be that these core competencies in pain management have only recently been published in 2013.41 Second, these core competencies provide general guidance related to pain medicine education for all major health care professionals and are not specific for medical education.41 Competency-based medical education is explicitly oriented to the desired performance characteristics of medical graduates that should largely be determined by stakeholders outside of the educational program.44,54 This review has highlighted the need for more defined assessment processes based on learning outcomes as defined by the IASP, which incorporate the interprofessional core competencies for pain management prelicensure health professionals (as described by Fishman et al.).41
4.1.2. Assessment for learning
It is generally accepted that assessments that include formative feedback can enhance learning.123 Although few studies in this review focussed on assessments aimed at driving learning, 3 studies described assessment methods that were used in a formative capacity to promote interprofessional learning.34,63,99 In these studies, a number of different health professionals worked together to integrate knowledge, skills, and attitudes to develop a pain management plan. The group sessions were facilitated by trained faculty staff during or after the education experience to provide useful opportunities for self-reflection and feedback.
Self-assessment of pain knowledge competencies and confidence in clinical capabilities was described in 15% of studies. Self-assessment and reflection methods are more suited to formative-type assessment, as self-reported abilities may vary from actual abilities and are susceptible to recall and social desirability biases.52 Two studies described methods (such as portfolio work and journals) that were shown to be useful for fostering habits of self-reflection and self-remediation.25,85 The portfolio work resulted in increased awareness of the depth of suffering experienced by patients in pain and enhanced a commitment to excellence in pain care. The reflective journals brought to light the fact that 1st-year medical students identify pain as a major concern in their early clinical experience and, in general, have a negative perception of pain medicine. Assessment for learning was shown to be an integral part of a pain medicine curriculum to foster emotional strength (eg, empathy, compassion, and caring), emotional intelligence (eg, awareness of the impact of pain on mood and perception of emotion in others), emotional resilience (eg, capacity for emotional self-repair and ability to tolerate frustration), and emotional regulation (eg, tolerance of difficulties and sense of professional duty).85
4.2. Appropriate methods of assessment
The diversity of methods used in this review reflects the range of the assessment purposes and settings. Written exercises, in particular, MCQs were the most common method used to assess medical students' knowledge and attitudes of pain medicine. This was not surprising because the current medical school environment with large student numbers and a pressurised curriculum favours the MCQ-type assessment.37 Multiple choice questions require less time to administer and mark than open-ended short answer questions and structured essays.49 Multiple choice questions have been shown to be more reliable in testing knowledge than traditional essay questions.5
True/false and single best answer-type questions were predominantly used to assess cognitive knowledge. More recently, medical schools are encouraged not to use true/false question formats because dichotomous choices are rare in medical practice, and it is difficult to frame true/false questions that are unambiguous and sufficiently challenging to discriminate between students.47 Questionnaires scored using a Likert rating scale were mostly used for assessing attitudes for research or for self-evaluation purposes rather than high-stakes summative assessments.
If well-constructed, MCQs can also be used to assess learning objectives that require higher-order thinking (such as applying knowledge to a clinical situation).28,37 Validity of MCQs can be increased by sampling the entirety of the domain and including sufficient numbers of items in the assessment instrument (in general, at least 30 items).28 The use of well-crafted items that address a range of difficulty levels and the careful selection of distractors to reduce the cueing effect MCQ testing will also increase validity of written tests.7,28 Extended-matching questions, in which the format of the question starts with a vignette and the student is required to make a diagnosis choosing an answer from a list of options, were not used in any studies in this review but have increasingly been used as a medical examination format.22,30 Because well-constructed MCQs and extended-matching questions are time-consuming to develop, there may be opportunities for agencies and universities to work collaboratively to construct a bank of questions related to pain medicine for medical educators to draw from, such as the Get Through Series.26,28,37
Higher-order assessment of medical students' pain competencies, for example, assessment of clinical skills, was undertaken using the OSCE format with simulated patients.55,56,73,101,108–110,127 The OSCE assessment was used effectively for assessing competencies such as attitude, empathy, and communication as part of a pain assessment in a variety of contexts (acute, low-back, and cancer pain).55,73,109,110,124,127 The OSCE may currently be the most appropriate summative assessment for educators to use to identify students who may be at risk of poor performance in a clinical workplace environment and is used extensively within clinical medical education.32,70,79,115 Careful attention needs to be paid to test content, design, and implementation factors such as wide sampling of cases, large numbers of stations and examiners, and good standardisation of patients to improve validity and reliability.119,124 This is especially relevant when the results will be used for high-stakes decision-making.119 Checklists have become standard in many OSCE programmes as was evidenced in this review, but global rating scales scored by experts (that show higher interstation reliability and better construct validity than do checklists) were only used in one study.119,124 Theoretical difficulties associated with OSCE-type assessments were discussed in 4 studies, such as the time taken to design an OSCE and deliver education for the actors, as well as the costs associated with actors. Three studies used volunteer patients, and one study used students as actors to avoid these costs, and it was felt that the positive effects of the intervention on patients and students validated the time spent by clinical staff. Debriefing is an essential component of simulation that allows the student to understand their decision-making processes and was discussed in 2 of the studies in this review.53 Examples of high-fidelity simulation experiences (involving SPs or computer-programmed mannequins) to broaden the assessment of pain-related core competencies for health professional students have been described elsewhere.13
Although the use of portfolios and multisource (360°) pain medicine assessments was poorly documented in this review, these methods have the potential for providing both formative and summative feedback on pain medicine competence.61 Multidimensional pain portfolios were described in one study to engage the students in active, structured reflection about pain-related experiences.85 Portfolios can be extended to encompass reporting on work done, progress made as well as goals and plans on how to further improve competence.29,89 Their format and content may differ considerably.29,89 Although these assessment methods can be time-consuming to prepare and review, they are most effective when goals and procedures are clearly communicated; there is a flexible structure underpinning the development of the portfolio; support is provided through mentoring, and there is robust integration of the assessment into the curriculum.29 Portfolios should ideally be judged by multiple assessors who should be encouraged to substantiate their judgments.89 Multisource (360°) assessments involving assessment by patients, peers, and other members of the clinical team have been shown to provide valuable insight into students' work habits and capacity for teamwork.4,31,32,38,58,117
Validity focusses on “whether a test actually succeeds in testing the competencies that it is designed to test.”124 Ideally, assessment of clinical competency should provide some measure of actual performance in a real-world environment and capacity to adapt to different environments.32 There were no published studies of direct observation of students' encounters with patients by supervising clinicians to assess the impact of pain medicine education on students' clinical performance. Demonstration of pain competencies in “real-world” settings favours formative assessment opportunities. These assessments require flexibility, as not all patients will provide adequate teaching/learning opportunities, and quality supervision is necessary to reduce patient risks.13 Refining of learning can take place during feedback to structure further learning.13 Creating valid and reliable assessment tools to predict real-world clinical performance of pain competencies is challenging, and further work is needed in this area to assist educators with this important task.32,54,124
This review has described multiple methods of pain medicine assessment in a variety of settings and contexts. It is clear from this review that there was no one gold standard for the assessment of the broad range of pain medicine competencies. Assessment methods had unique strengths and flaws. The use of multiple approaches over time can compensate for deficits in any one method.32 Three studies stood out as examples of multifaceted evaluation of the effectiveness of pain education modules by assessing the 3 main domains of learning, that is, knowledge, attitudes, and skills.9,63,85 Further work is needed to encourage medical schools to implement similar multidimensional assessments and adapt the tools to their unique local situations.
4.3. Implications for practice and future research
Only 5 studies included examination of students' empathy towards patients with pain.46,73,85,87,95 This review suggests that little attention is being paid to documented assessment of students' ability to display empathy, considering that the development of compassionate and empathetic practices has been considered a critically important aspect of pain management.14
Different approaches to assessment of interprofessional skills were explored in this review, such as medical student/nursing student dyads in a clinical simulation experience, case-based workshops for medical and nursing students, and group assignments (medical, nursing, dentistry, pharmacy, physical therapy, and occupational therapy students).34,63,99 It is widely accepted that to maximize optimal pain management, health professionals need to work together in interprofessional teams.15,42,45,64,97 Continued work is needed to develop and refine formal assessment of interprofessional pain knowledge, attitudes, and clinical skills to enhance the seriousness and importance of this aspect of pain medicine education.20,81
This review has shown that there is a lack of attention being paid to exploring assessment of topics such as psychological and physical therapy approaches to pain management, as well as types of pain such as headache, visceral pain, and chronic primary pain. The needs of special populations such as children, the elderly, developmentally challenged, and patients with opioid tolerance were also neglected. More than half of the studies in this review were focussed on opioid therapy rather than on evidence-based advances in pharmacological management of pain, such as the use of tricyclic antidepressants, selective noradrenaline reuptake inhibitors, antiepileptics, topical lidocaine, and capsaicin.39 These are topics listed in the IASP Curriculum Outline on Pain for Medicine, which should be covered if medical graduates are to be adequately prepared for the management of pain.67 Emerging concepts in pain theory, such as central sensitisation, neuroplasticity, allodynia, cortical modulation, nociception being neither sufficient nor necessary to evoke pain, and the concept of pain as a disease entity were seldom included in the assessment of pain medicine competency.3,113
Assessment tools need to be kept up to date as scientific knowledge grows. A clear example of assessment tools becoming outdated was demonstrated in the area of opioid prescription for pain. Earlier studies (1998–2008) focussed on opiophobia, or the reluctance of medical students to prescribe opioids for patients in pain.50,76,80,107,128 The current teaching at that time was that patients had the right to pain relief, and that the risks of opioid prescription in terms of dependency and overdose were low.93,128 Pain experts are now warning against the liberal use of opioids especially for chronic non-cancer pain (CNCP).77 Medical schools have recently incorporated core competencies into their curricula such as safe prescribing for acute short-term opioid use and the management of long-term opioid prescriptions for the treatment of CNCP and have committed to assessing students' competence in these areas.11
4.3.1. How should pain medicine competencies be assessed?
Current assessment practices can be enhanced by the recognition and application of the principles summarised in Box 2.32,33,123,124 Ideally, assessment of pain medicine competencies requires a purposely built combination of individual instruments, each being optimally designed in terms of assessment for learning and assessment of learning.102
Principles of effective assessment of pain medicine competencies
Be clear on the purpose of assessment
Provide timely feedback and mentoring
Ensure continuous and frequent assessments (formative and summative)
Incorporate multiple methods of assessments in a variety of contexts
Define the endpoint of assessment and set appropriate standards
Plan assessments using a developmental perspective in terms of levels of learning
Ensure that assessments are grounded on evidence-based content
Design authentic work-based assessments
4.3.2. Framework for assessments
This review has highlighted a lack of comprehensive pain medicine assessment models. In part, this reflects the low priority given by medical schools to the delivery of pain medicine education internationally. In addition, detailed pain medicine competencies specifically designed for medical students have yet to be developed. The resources available at present are the pain management core competencies for prelicensure health professionals (Fishman et al.),41 which are designed to be used in conjunction with the IASP curriculum outline on pain for medical students.
Pain medicine is a complex and multifaceted discipline that is mostly taught as a topic integrated throughout the entire medical curriculum. An assessment framework would, therefore, be useful to provide a structured conceptual map of essential content components and core competencies for assessment in the pain medicine curriculum as a whole. Hence, a Pain Medicine Assessment Framework (Table 5) has been developed to provide a systematic and comprehensive approach to designing pain medicine assessments for medical students.33,67 This framework incorporates the assessment of pain management core competencies recommended for prelicensure health professionals as described by Fishman et al.41 and the IASP curriculum (appendix 1 and 2, available at http://links.lww.com/PR9/A36). The framework can also act as a point of reference for the various members of the pain medicine education community and can be used as a blueprint for universities wishing to incorporate pain medicine into the curriculum. It allows for planning and recording of continuous and frequent assessments, and the sequencing of assessments in terms of level of learning. It ensures that there is consistent assessment across the spectrum of clinical contexts and reduces bias in assessment practice by combining the various methods of assessments. The Pain Medicine Assessment Framework is a useful record of assessment for the student because it encourages self-reflection, but it is also a valuable resource for supporting critical reflection and evaluation of the pain medicine curriculum by medical educators.
Pain medicine is embedded into the discipline structure of medicine. It is therefore likely that pain medicine assessment occurs within medical schools but remains unpublished and as a result was not included in this review. Second, although many studies described individual assessment methods, quality data that evaluated and compared assessment methods were lacking.
This review highlights the diversity of instruments used to assess the outcomes of pain medicine education and competencies in medical students. There is no one “gold standard” method for assessing knowledge, attitudes, and skills of pain medicine. In general, the studies reviewed did not reflect contemporary theories of educational assessment.16 Most assessments were performed for low-stakes summative purposes and were not constructed using specific pain competencies or learning objectives. Assessments were predominantly undertaken using written tests or clinical simulation methods. Important topics that are intrinsic in the pain medicine curriculum were neglected, and current evidence-based pain medicine knowledge was not reflected in many assessments. Some medical schools demonstrated innovative assessment methods (such as reflective journals, vignettes, and portfolios) and multifaceted assessment processes to measure multiple domains of competence in clinical pain medicine practice. Future research could prioritise the development of robust assessment tools that can effectively measure the abilities of medical students to integrate pain-related knowledge, skills, and attitudes into clinical practice.
Assessment plays an integral role in competency-based education and should evolve in concert with delivery of the educational objectives. A framework of assessment has been developed to encourage systematic planning of pain medicine assessment at medical schools internationally to validate the objectives of the curriculum and to provide effective feedback to the students. Importantly, this framework will promote fundamental aspects of competency-based pain medicine assessment such as continuous multidimensional assessments in a variety of clinical contexts based on well-defined pain medicine competencies.
The authors declare that they have no competing interests.
Author contributions: E.E. Shipton: study concept and Pain Medicine Assessment Framework design, literature search, data acquisition, analysis and interpretation of data, and drafting of manuscript. E.J. Visser, C. Steketee, F. Bate: study concept, analysis and interpretation of data, and drafting of manuscript. All authors read and approved the final manuscript.
Appendix A. Supplemental digital content
Supplemental digital content associated with this article can be found online at http://links.lww.com/PR9/A36.
. Accreditation council for graduate medical education
core competencies. Available at: https://www.ecfmg.org/echo/acgme-core-competencies.html
. Accessed March 23, 2018.
. Educating future physicians in palliative and end-of-life care (EFPPEC): Core competencies/learning outcomes in palliative and end-of-life care. Available at: https://www.cspcp.ca/wp-content/uploads/2017/03/EFPPEC_UG_Comp.pdf
. Accessed March 23, 2018.
. Adillon C, Lozano E, Salvat I. Comparison of pain neurophysiology knowledge among health sciences students: a cross-sectional study. BMC Res Notes 2015;8:592.
. Adwan J. Dynamic online peer evaluations to improve group assignments in nursing e-learning environment. Nurse Educ Today 2016;41:67–72.
. Al-Rukban MO. Guidelines for the construction of multiple choice questions tests. J Fam Community Med 2006;13:125–33.
. Ali N, Thomson D. A comparison of the knowledge of chronic pain and its management between final year physiotherapy and medical students. Eur J Pain 2009;13:38–50.
. Ali SH, Carr PA, Ruit KG. Validity and reliability of scores obtained on multiple-choice questions: why functioning distractors matter. J Scholarship Teach Learn 2016;16:1–14.
. American Board of Pain Medicine
. Definition of pain medicine
. Available at: http://www.abpm.org/what
. Accessed July 20, 2017.
. Ameringer S, Fisher D, Sreedhar S, Ketchum JM, Yanni L. Pediatric pain management education in medical students: impact of a web-based module. J Palliat Med 2012;15:978–83.
. Anderson MB, Kanter SL. Medical education
in the United States and Canada, 2010. Academic medicine. J Assoc Am Med Colleges 2010;85(9 suppl):S2–S18.
. Antman KH, Berman HA, Flotte TR, Flier J, Dimitri DM, Bharel M. Developing core competencies for the prevention and management of prescription drug misuse: a medical education
collaboration in Massachusetts. Acad Med 2016;91:1348–51.
. Argyra E, Siafaka I, Moutzouri A, Papadopoulos V, Rekatsina M, Vadalouca A, Theodoraki K. How does an undergraduate pain course influence future physicians' awareness of chronic pain concepts? A comparative study. Pain Med 2015;16:301–11.
. Arwood E, Rowe JM, Singh NS, Carr DB, Herr KA, Chou R. Implementing a paradigm shift: incorporating pain management competencies into pre-licensure curricula. Pain Med 2015;16:291–300.
. Banja JD. Toward a more empathic relationship in pain medicine
. Pain Med 2008;9:1125–9.
. Barr H, Koppel I, Reeves S, Hammick M, Freeth DS. Effective interprofessional education: Argument, assumption and evidence (promoting partnership for health). Oxford, United Kingdom: John Wiley & Sons, 2008.
. Biggs J. Constructive alignment in university teaching. HERDSA Rev Higher Education 2014;1:5–22.
. Breivik H, Collett B, Ventafridda V, Cohen R, Gallacher D. Survey of chronic pain in Europe: prevalence, impact on daily life, and treatment. Eur J Pain 2006;10:287–333.
. Briggs AM, Slater H, Smith AJ, Parkin-Smith GF, Watkins K, Chua J. Low back pain-related beliefs and likely practice behaviours among final-year cross-discipline health students. Eur J Pain 2013;17:766–75.
. Briggs EV, Battelli D, Gordon D, Kopf A, Ribeiro S, Puig MM, Kress HG. Current pain education within undergraduate medical studies across Europe: advancing the provision of pain education and learning (APPEAL) study. BMJ Open 2015;5:e006984.
. Carr E, Watt-Watson J. Interprofessional pain education: definitions, exemplars and future directions. Br J Pain 2012;6:59–65.
. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from Flexner to competencies. Acad Med 2002;77:361–7.
. Case SM, Swanson DB, Ripkey DR. Comparison of items in five-option and extended-matching formats for assessment
of diagnostic skills. Acad Med 1994;69(10 suppl):S1–S3.
. Charlton E. Core curriculum for professional education. Seattle: IASP, 2005.
. Cohen IT, Bennett L. Introducing medical students to paediatric pain management. Med Educ 2006;40:476.
. Corrigan C, Desnick L, Marshall S, Bentov N, Rosenblatt RA. What can we learn from first—year medical students' perceptions of pain in the primary care setting? Pain Med 2011;12:1216–22.
. CRC Press. Book series get through. Available at: https://www.crcpress.com/Get-Through/book-series/CRCGETTHROUG
. Accessed April 27, 2018.
. Dominick C, Blyth F, Nicholas M. Patterns of chronic pain in the New Zealand population. N Z Med J 2011;124:63–76.
. Downing SM, Haladyna TM. Validity threats: overcoming interference with proposed interpretations of assessment
data. Med Educ 2004;38:327–33.
. Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education
: why do they meet with mixed success? A systematic review
. Med Educ 2007;41:1224–33.
. Eijsvogels TMH, van den Brand TL, Hopman MTE. Multiple choice questions are superior to extended matching questions to identify medicine and biomedical sciences students who perform poorly. Perspect Med Educ 2013;2:252–63.
. Emke AR, Cheng S, Dufault C, Cianciolo AT, Musick D, Richards B, Violato C. Developing professionalism via multisource feedback in team-based learning. Teach Learn Med 2015;27:362–5.
. Epstein RM. Assessment
in medical education
. N Engl J Med 2007;356:387–96.
. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA 2002;287:226–35.
. Erickson JM, Brashers V, Owen J, Marks JR, Thomas SM. Effectiveness of an interprofessional workshop on pain management for medical and nursing students. J Interprof Care 2016;30:466–74.
. Fabbian F, De Giorgi A, Pala M, Mallozzi Menegatti A, Gallerani M, Manfredini R. Pain prevalence and management in an internal medicine setting in Italy. Pain Res Treat 2014;2014:628284.
. Faculty of Pain Medicine
ANZCA. About FPM. Available at: http://fpm.anzca.edu.au/about-fpm
. Accessed May 11, 2017.
. Ferris H, O'Flynn D. Assessment
in medical education
; what are we trying to achieve? Int J Higher Educ 2015;4:139–44.
. Fete MG, Haight RC, Clapp P, McCollum M. Peer evaluation instrument development, administration, and assessment
in a team-based learning curriculum. Am J Pharm Educ 2017;81:68.
. Finnerup NB, Attal N, Haroutounian S, McNicol E, Baron R, Dworkin RH, Gilron I, Haanpää M, Hansson P, Jensen TS, Kamerman PR, Lund K, Moore A, Raja SN, Rice AS, Rowbotham M, Sena E, Siddall P, Smith BH, Wallace M. Pharmacotherapy for neuropathic pain in adults: a systematic review
and meta-analysis. Lancet Neurol 2015;14:162–73.
. Fishman SM, Young HM. Driving needed change in pain education. Pain Med 2016;17:1790–2.
. Fishman SM, Young HM, Lucas Arwood E, Chou R, Herr K, Murinson BB, Watt-Watson J, Carr DB, Gordon DB, Stevens BJ. Core competencies for pain management: results of an interprofessional consensus summit. Pain Med 2013;14:971–81.
. Flor H, Turk DC. Chronic pain: an intergrated biobehavioural approach. Seattle: IASP Press, 2011.
. Frank JR, Snell LS, Cate OT, Holmboe ES, Carraccio C, Swing SR, Harris P, Glasgow NJ, Campbell C, Dath D, Harden RM, Iobst W, Long DM, Mungroo R, Richardson DL, Sherbino J, Silver I, Taber S, Talbot M, Harris KA. Competency-based medical education
: theory to practice. Med Teach 2010;32:638–45.
. Frenk J, Chen L, Bhutta ZA, Cohen J, Crisp N, Evans T, Fineberg H, Garcia P, Ke Y, Kelley P, Kistnasamy B, Meleis A, Naylor D, Pablos-Mendez A, Reddy S, Scrimshaw S, Sepulveda J, Serwadda D, Zurayk H. Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. Lancet 2010;376:1923–58.
. Gatchel RJ, Okifuji A. Evidence-based scientific data documenting the treatment and cost-effectiveness of comprehensive pain programs for chronic nonmalignant pain. J Pain 2006;7:779–93.
. Gehlhar K, Tauschel D, Lux EA, Junker U. Teaching pain management. An innovative curriculum model at the University of Witten/Herdecke (UWH) [in German]. Schmerz 2011;25:45–54.
. Georgiades S, Papageorgiou A, Perdikogianni M, McCrorie P. Psychology for psychologists: A problem based approach to undergraduate psychology teaching. London, United Kingdom: Palgrave Macmillan UK, 2015.
. Ger LP, Lee MC, Wong CS, Chao SS, Wang JJ, Ho ST. The effect of education and clinical practice on knowledge enlightenment to and attitudes toward the use of analgesics for cancer pain among physicians and medical students. Acta Anaesthesiol Sin 2003;41:105–14.
. Gerbershagen HJ, Aduckathil S, van Wijck AJ, Peelen LM, Kalkman CJ, Meissner W. Pain intensity on the first day after surgery: a prospective cohort study comparing 179 surgical procedures. Anesthesiology 2013;118:934–44.
. Gordon MJ. A review
of the validity and accuracy of self-assessments in health professions training. Acad Med 1991;66:762–9.
. Green M, Tariq R, Green P. Improving patient safety through simulation training in anesthesiology: where are we? Anesthesiol Res Pract 2016;2016:4237523.
. Gruppen LD, Mangrulkar RS, Kolars JC. The promise of competency-based education
in the health professions for improving global health. Hum Resour Health 2012;10:43.
. Hanna MN, Donnelly MB, Montgomery CL, Sloan PA. Perioperative pain management education: a short structured regional anesthesia course compared with traditional teaching among medical students. Reg Anesth Pain Med 2005;30:523–8.
. Haq I, Fuller J, Dacre J. The use of patient partners with back pain to teach undergraduate medical students. Rheumatology (Oxford) 2006;45:430–4.
. Harden RM. AMEE guide no. 14: outcome-based education: part 1-an introduction to outcome-based education. Med Teach 1999;21:7–14.
. Harendza S, Berberat P, Kadmon M. Assessing competences in medical students with a newly designed 360-degree examination of a simulated first day of residency: a feasibility study. J Commun Med Health Educ 2017;7:4.
. Harlen W. Teachers' summative practices and assessment
for learning–tensions and synergies. Curriculum J 2005;16:207–23.
. Hirsh AT, Hollingshead NA, Bair MJ, Matthias MS, Kroenke K. Preferences, experience, and attitudes in the management of chronic pain and depression: a comparison of physicians and medical students. Clin J Pain 2014;30:766–74.
. Holmboe ES, Durning SJ, Hawkins RE. Practical guide to the evaluation of clinical competence e-book. Philadelphia, PA: Elsevier Health Sciences, 2017.
. Hoy D, Bain C, Williams G, March L, Brooks P, Blyth F, Woolf A, Vos T, Buchbinder R. A systematic review
of the global prevalence of low back pain. Arthritis Rheum 2012;64:2028–37.
. Hunter J, Watt-Watson J, McGillion M, Raman-Wilms L, Cockburn L, Lax L, Stinson J, Cameron A, Dao T, Pennefather P, Schreiber M, Librach L, Kavanagh T, Gordon A, Cullen N, Mock D, Salter M. An interfaculty pain curriculum: lessons learned from six years experience. PAIN 2008;140:74–86.
. Hunter JP, Stinson J, Campbell F, Stevens B, Wagner SJ, Simmons B, White M, van Wyk M. A novel pain interprofessional education strategy for trainees: assessing impact on interprofessional competencies and pediatric pain knowledge. Pain Res Manag 2015;20:e12–e20.
. Institute of Medicine (US) Committee on Advancing Pain Research Care, and Education. Relieving pain in America: a blueprint for transforming prevention, care, education, and research. Washington, D.C.: National Academies Press, 2011.
. International Association for the Study of Pain. About IASP. Available at: http://www.iasp-pain.org/AboutIASP/?navItemNumber=506
. Accessed July 3, 2018.
. International Association for the Study of Pain. IASP curriculum outline on pain for medicine. Available at: http://www.iasp-pain.org/Education/CurriculumDetail.aspx?ItemNumber=729
. Accessed March 21, 2018.
. Kaki AM. Medical students' knowledge and attitude toward cancer pain management in Saudi Arabia. Saudi Med J 2011;32:628–32.
. Kennedy N, Healy J, O'Sullivan K. The beliefs of third-level healthcare students towards low-back pain. Pain Res Treat 2014;2014:675915.
. Khan KZ, Ramachandran S, Gaunt K, Pushkar P. The objective structured clinical examination (OSCE): AMEE guide no. 81. Part I: an historical and theoretical perspective. Med Teach 2013;35:e1437–1446.
. Lasch K, Greenhill A, Wilkes G, Carr D, Lee M, Blanchard R. Why study pain? A qualitative analysis of medical and nursing faculty and students' knowledge of and attitudes to cancer pain management. J Palliat Med 2002;5:57–71.
. Lee A, Steketee C, Rogers G, Moran M. Towards a theoretical framework for curriculum development in health professional education. Focus Health Prof Education 2013;14:70–83.
. Leila NM, Pirkko H, Eeva P, Eija K, Reino P. Training medical students to manage a chronic pain patient: both knowledge and communication skills are needed. Eur J Pain 2006;10:167–70.
. Lockyer J, Carraccio C, Chan MK, Hart D, Smee S, Touchie C, Holmboe ES, Frank JR. Core principles of assessment
in competency-based medical education
. Med Teach 2017;39:609–16.
. Loeser JD, Schatman ME. Chronic pain management in medical education
: a disastrous omission. Postgrad Med 2017;129:332–5.
. Manalo MFC. Knowledge toward cancer pain and the use of opioid analgesics among medical students in their integrated clinical clerkship. Palliat Care Res Treat 2008;2:9–17.
. Manchikanti L, Kaye AM, Knezevic NN, McAnally H, Slavin K, Trescot AM, Blank S, Pampati V, Abdi S, Grider JS, Kaye AD, Manchikanti KN, Cordner H, Gharibo CG, Harned ME, Albers SL, Atluri S, Aydin SM, Bakshi S, Barkin RL, Benyamin RM, Boswell MV, Buenaventura RM, Calodney AK, Cedeno DL, Datta S, Deer TR, Fellows B, Galan V, Grami V, Hansen H, Helm li S, Justiz R, Koyyalagunta D, Malla Y, Navani A, Nouri KH, Pasupuleti R, Sehgal N, Silverman SM, Simopoulos TT, Singh V, Solanki DR, Staats PS, Vallejo R, Wargo BW, Watanabe A, Hirsch JA. Responsible, safe, and effective prescription of opioids for chronic non-cancer pain: American Society of Interventional Pain Physicians (ASIPP) Guidelines. Pain Physician 2017;20:S3–s92.
. MBF Foundation. The high price of pain: the economic impact of persistent pain in Australia. Available at: https://www.bupa.com.au/staticfiles/BupaP3/Health%20and%20Wellness/MediaFiles/PDFs/MBF_Foundation_the_price_of_pain.pdf
. Accessed May 1, 2018.
. Medical Council of Canada. OSCE: definition, purpose and format. Available at: https://mcc.ca/examinations/osce-orientation/definition/
. Accessed April 25, 2018.
. Merrill JM, Hill CS, Laux LM, Lorimor RI, Thornby JI, Thorpe D, Weinstein SM. Measuring medical students' reluctance to prescribe opioids for cancer pain. Psychol Rep 1999;84:28–30.
. Mezei L, Murinson BB, Johns Hopkins Pain Curriculum Development T. Pain education in North American medical schools. J Pain 2011;12:1199–208.
. Miller GE. The assessment
of clinical skills/competence/performance. Acad Med 1990;65(9 suppl):S63–S67.
. Morley-Forster P, Karpinski J. Pain medicine
—a new credential in Canada. Pain Med 2015;16:1038–44.
. Morris H, Ryan C, Lauchlan D, Field M. Do medical student attitudes towards patients with chronic low back pain improve during training? a cross-sectional study. BMC Med Educ 2012;12:10.
. Murinson BB, Nenortas E, Mayer RS, Mezei L, Kozachik S, Nesbit S, Haythornthwaite JA, Campbell JN. A new program in pain medicine
for medical students: integrating core curriculum knowledge with emotional and reflective development. Pain Med 2011;12:186–95.
. Nabishah M, Nasri A, Salam A, Ima Nirwana S. Standard setting for assessment
of basic medical science modules. Proced Social Behav Sci 2011;18:118–21.
. Niemi-Murola L, Nieminen JT, Kalso E, Poyhia R. Medical undergraduate students' beliefs and attitudes toward pain: how do they mature? Eur J Pain 2007;11:700–6.
. Nimmaanrat S, Oofuvong M. Attitudes of medical students regarding cancer pain management: comparison between pre- and post-lecture test findings. Asian Pac J Cancer Prev 2015;16:7453–6.
. Oudkerk Pool A, Govaerts MJB, Jaarsma D, Driessen EW. From aggregation to interpretation: how assessors judge complex data in a competency-based portfolio. Adv Health Sci Educ 2018;23:275–87.
. Paneduro D, Pink LR, Smith AJ, Chakraborty A, Kirshen AJ, Backstein D, Woods NN, Gordon AS. Development, implementation and evaluation of a pain management and palliative care educational seminar for medical students. Pain Res Manag 2014;19:230–4.
. Pilowsky I. An outline curriculum on pain for medical schools. PAIN 1988;33:1–2.
. Plymale MA, Sloan PA, Johnson M, LaFountain P, Snapp J, Sloan DA. Cancer pain education: the use of a structured clinical instruction module to enhance learning among medical students. J Pain Symptom Manage 2000;20:4–11.
. Portenoy RK, Foley KM. Chronic use of opioid analgesics in non-malignant pain: report of 38 cases. PAIN 1986;25:171–86.
. Pöyhiä R, Niemi-Murola L, Kalso E. The outcome of pain related undergraduate teaching in Finnish medical faculties. PAIN 2005;115:234–7.
. Puljak L, Sapunar D. Web-based elective courses for medical students: an example in pain. Pain Med 2011;12:854–63.
. Rainville J, Bagnall D, Phalen L. Health care providers' attitudes and beliefs about functional impairments and chronic back pain. Clin J Pain 1995;11:287–95.
. Reeves S, Perrier L, Goldman J, Freeth D, Zwarenstein M. Interprofessional education: effects on professional practice and healthcare outcomes (update). Cochrane Database Syst Rev 2013;CD002213.
. Royal College of Anaesthetists. Faculty of pain medicine
of the Royal College of Anaesthetists. Available at: https://www.rcoa.ac.uk/faculty-of-pain-medicine/
. Accessed May 15, 2018.
. Salam T, Saylor JL, Cowperthwait AL. Attitudes of nurse and physician trainees towards an interprofessional simulated education experience on pain assessment
and management. J Interprof Care 2015;29:276–8.
. Schopflocher D, Taenzer P, Jovey R. The prevalence of chronic pain in Canada. Pain Res Manag 2011;16:445–50.
. Schreiner U, Haefner A, Gologan R, Obertacke U. Effective teaching modifies medical student attitudes toward pain symptoms. Eur J Trauma Emerg Surg 2011;37:655–9.
. Schuwirth LW, van der Vleuten CP. Changing education, changing assessment
, changing research? Med Educ 2004;38:805–12.
. Shipton EE, Bate F, Garrick R, Steketee C, Shipton EA, Visser EJ. Systematic review
of pain medicine
content, teaching, and assessment
in medical school curricula internationally. Pain Ther 2018;7:139–61.
. Shipton EE, Bate F, Garrick R, Steketee C, Visser EJ. Pain medicine
content, teaching and assessment
in medical school curricula in Australia and New Zealand. BMC Med Educ 2018;18:110.
. Siddall PJ, Cousins MJ. Persistent pain as a disease entity: implications for clinical management. Anesth Analg 2004;99:510–20. Table of contents.
. Silva EJdGDGd, Dixe MdACR. Prevalência e características de dor em pacientes internados em hospital português. Revista Dor 2013;14:245–50.
. Sloan PA, Montgomery C, Musick D. Medical student knowledge of morphine for the management of cancer pain. J Pain Symptom Manage 1998;15:359–64.
. Sloan PA, Plymale M, LaFountain P, Johnson M, Snapp J, Sloan DA. Equipping medical students to manage cancer pain: a comparison of three educational methods. J Pain Symptom Manage 2004;27:333–42.
. Sloan PA, Plymale MA, Johnson M, Vanderveer B, LaFountain P, Sloan DA. Cancer pain management skills among medical students: the development of a cancer pain objective structured clinical examination. J Pain Symptom Manage 2001;21:298–306.
. Stevens DL, King D, Laponis R, Hanley K, Zabar S, Kalet AL, Gillespie C. Medical students retain pain assessment
and management skills long after an experiential curriculum: a controlled study. PAIN 2009;145:319–24.
. Strohbuecker B, Mayer H, Evers GCM, Sabatowski R. Pain prevalence in hospitalized patients in a German University Teaching Hospital. J Pain Symptom Manag 2005;29:498–506.
. Symonds TL, Burton AK, Tillotson KM, Main CJ. Do attitudes and beliefs influence work loss due to low back trouble? Occup Med (Lond) 1996;46:25–32.
. Tauben DJ, Loeser JD. Pain education at the University of Washington School of Medicine. J Pain 2013;14:431–7.
. Tellier PP, Belanger E, Rodriguez C, Ware MA, Posel N. Improving undergraduate medical education
about pain assessment
and management: a qualitative descriptive study of stakeholders' perceptions. Pain Res Manag 2013;18:259–65.
. Terry R, Hing W, Orr R, Milne N. Do coursework summative assessments predict clinical performance? a systematic review
. BMC Med Educ 2017;17:40.
. The American Academy of Pain Medicine
. The voice of pain medicine
. Available at: http://www.painmed.org/
. Accessed May 15, 2018.
. Thomson FC, MacKenzie RK, Anderson M, Denison AR, Currie GP. Incorporating patient partner scores into high stakes assessment
: an observational study into opinions and attitudes. BMC Med Educ 2017;17:214.
. Trinca J. Knowledge of pain mechanisms and management in recent medical graduates. Sydney: University of Sydney, 1998.
. Turner JL, Dankoski ME. Objective structured clinical exams: a critical review
. Fam Med 2008;40:574–8.
. Twigg OC, Byrne DG. The influence of contextual variables on judgments about patients and their pain. Pain Med 2015;16:88–98.
. Ussher WBG, Earl K. “Summative” and “formative”: Confused by the assessment
terms? New Zealand Journal of Teachers' Work 2010; 7:53–63.
. van den Beuken-van Everdingen MH, de Rijke JM, Kessels AG, Schouten HC, van Kleef M, Patijn J. Prevalence of pain in patients with cancer: a systematic review
of the past 40 years. Ann Oncol 2007;18:1437–49.
. van der Vleuten CP, Schuwirth LW, Driessen EW, Dijkstra J, Tigelaar D, Baartman LK, van Tartwijk J. A model for programmatic assessment
fit for purpose. Med Teach 2012;34:205–14.
. Wass V, Van der Vleuten C, Shatzer J, Jones R. Assessment
of clinical competence. Lancet 2001;357:945–9.
. Watt-Watson J, Hunter J, Pennefather P, Librach L, Raman-Wilms L, Schreiber M, Lax L, Stinson J, Dao T, Gordon A, Mock D, Salter M. An integrated undergraduate pain curriculum, based on IASP curricula, for six health science faculties. PAIN 2004;110:140–8.
. Watt-Watson J, Peter E, Clark AJ, Dewar A, Hadjistavropoulos T, Morley-Forster P, O'Leary C, Raman-Wilms L, Unruh A, Webber K. The ethics of Canadian entry-to-practice pain competencies: how are we doing? Pain Res Manag 2013;18:25.
. Weiner DK, Morone NE, Spallek H, Karp JF, Schneider M, Washburn C, Dziabiak MP, Hennon JG, Elnicki DM. E-learning module on chronic low back pain in older adults: evidence of effect on medical student objective structured clinical examination performance. J Am Geriatr Soc 2014;62:1161–7.
. Weinstein SM, Laux LF, Thornby JI, Lorimor RJ, Hill CS Jr, Thorpe DM, Merrill JM. Medical students' attitudes toward pain and the use of opioid analgesics: implications for changing medical school curriculum. South Med J 2000;93:472–8.
. Weissman DE, Dahl JL. Attitudes about cancer pain: a survey of Wisconsin's first-year medical students. J Pain Symptom Manage 1990;5:345–9.
. Wijnen-Meijer M, Burdick W, Alofs L, Burgers C, ten Cate O. Stages and transitions in medical education
around the world: clarifying structures and terminology. Med Teach 2013;35:301–7.
. Wilson A, Feyer A. Review
of medical intern training: Australian Health Ministers' Advisory Council. www.coaghealthcouncil.gov.au/medicalinternreview
. Accessed August 11, 2018.
. Yanni LM, Priestley JW, Schlesinger JB, Ketchum JM, Johnson BA, Harrington SE. Development of a comprehensive e-learning resource in pain management. Pain Med 2009;10:95–105.