Impact of a Musculoskeletal “Mini-Residency” Professional Development Program on Knee Magnetic Resonance Imaging Orders by Primary Care Providers : JCR: Journal of Clinical Rheumatology

Secondary Logo

Journal Logo

Original Articles

Impact of a Musculoskeletal “Mini-Residency” Professional Development Program on Knee Magnetic Resonance Imaging Orders by Primary Care Providers

Mulcaire-Jones, Erica MD; Barker, Andrea M. MPAS, PA-C†,‡; Beck, J. Peter MD§; Lawrence, Phillip PharmD; Cannon, Grant W. MD†,¶; Battistone, Michael J. MD†,¶

Author Information
JCR: Journal of Clinical Rheumatology: August 2022 - Volume 28 - Issue 5 - p 245-249
doi: 10.1097/RHU.0000000000001842

Abstract

With the exception of back complaints, knee pain is the most common self-reported chronic joint condition; across all age groups, it is more prevalent than symptoms of all other joints combined.1 A 2006 analysis of data from the National Health and Nutrition Examination Survey showed that of people aged 63 to 90 years, 37% had radiographic knee osteoarthritis (OA).2 In addition, the prevalence of knee pain has doubled in women and tripled in men in the past 20 years, independent of age and obesity.3

Most patients with knee pain seek evaluation from health care providers in primary care clinics as access to musculoskeletal (MSK) specialists is limited.4 Unfortunately, gaps in MSK instruction have been recognized across the continuum of medical training and throughout the range of health professions education (HPE).4–12 These gaps contribute to inefficiency in health care systems, and overutilization of magnetic resonance imaging (MRI) has been identified as a significant component of unnecessary cost.13–16 Compared with orthopedists, primary care providers (PCPs) are more likely to order a knee MRI without documenting significant examination or x-ray findings to justify this procedure, often finding OA that might have been established by less expensive imaging.13–15 In a retrospective analysis of knee MRI orders placed by PCPs for patients older than 40 years with knee pain, orthopedists would have ordered only 12 of the 100 studies ordered by PCPs, given the information available at the time the order was placed.13 We believe that MRI overutilization reflects an educational need rooted in an inadequate MSK training for PCPs; several articles calling for effective interventions to address this educational need have recently been published.13–15,17–19 There has not been a similar comparison between orthopedists and rheumatologists. While a recent study of Medicare claims data found that orthopedists order more MRI scans than rheumatologists, the specific joints examined were not specified, and the clinical context was not described.20

We have previously described a continuing professional development (CPD) program developed at the George E. Wahlen Salt Lake City (SLC) Veterans Affairs Medical Center, which was cost-effective in changing clinical practice behaviors of PCPs—participants demonstrated an increase in the number of joint injections they performed following training.21,22 This phase of our analysis examines whether this educational experience also impacted PCPs' MRI utilization rates.

MATERIALS AND METHODS

Setting and Participants

Records of 26 PCPs who participated in the SLC MSK Mini-Residency between April 2012 and October 2014 were included in the data analysis. Participants were health care providers within the Veterans Affairs (VA) system who self-selected to participate in this weeklong (40 hours) accredited CPD program. This was an intensive experience; to provide a high faculty-to-student ratio, each offering of the SLC MSK Mini-Residency was limited to 4 participants. The program schedule is shown in the Supplemental Information (https://links.lww.com/RHU/A448).21 On the first day, faculty demonstrated a systematic examination of the knee. Participants were provided a checklist of the examination elements, which they used as a memory aid in a peer learning session, performing the examination on one another under faculty observation. Following this exercise, common causes of knee pain were presented didactically. Because of the high prevalence of OA in patients with knee pain, we emphasized the importance of evaluating patients with conventional x-rays. We also highlighted the high correlation between knee OA and meniscal disease (including meniscal tears) and the limited benefit of arthroscopy over physical therapy in treating most patients with knee OA.23

On the final day of the course, learners participated in an objective structured clinical examination (OSCE) in which they were asked to evaluate a simulated patient (M.J.B.) presenting with an acute meniscal injury in the setting of undiagnosed knee OA. Participants were provided with a written summary of the history and then performed elements of the physical examination that they had recently learned. Considering what had been taught earlier in the course, it was anticipated that they would order weight-bearing x-rays and recognize that knee MRI was unnecessary in light of the radiographic findings of OA (in which meniscal pathology is implied).24,25 Clinical reasoning was explored in a semistructured interview through several questions: “What do you think is the most likely explanation for this patient's knee pain? What would you do now? What technique (e.g., weight-bearing) would you specify in your x-ray order? Would MRI be useful? Why or why not?” Immediate feedback—emphasizing that MRI would not provide additional information needed for clinical reasoning or decision-making—was given to participants by the 2 faculty (A.M.B., M.J.B.) who conducted the OSCE. Feedback techniques used in the OSCE reflected those recommended by Hewson and Little26—feedback was focused and specific, based on behaviors observed during the OSCE, delivered with a respectful, nonjudgmental approach—and concluded by soliciting feedback from each participant regarding their thoughts and feelings about the experience and how their learning might be further supported. A more detailed description of the OSCE, including its development and validity evidence supporting its use as a capstone element in educational experiences, has been published.27

Data Collection Protocol

The “training date” was defined as the date a participant began the Mini-Residency; this was identified using program records. All knee MRI orders placed by each participant within 1 year prior to and 1 year after their training date were identified through the VA Corporate Data Warehouse. Within the study period, there was no policy change within our local VA system requiring knee x-ray prior to MRI and no orthopedic service agreement requiring MRI prior to referral for knee pain.

MRI Order Classification Scheme

Knee MRI orders were classified into 3 categories: “inappropriate,” “probably inappropriate,” and “possibly appropriate” (Fig. 1), based on the American College of Radiology Appropriateness Criteria.28 Orders were classified as “inappropriate” if weight-bearing knee x-rays had not been completed within the 12 months preceding the MRI order. Orders were classified as “probably inappropriate” if weight-bearing x-rays had been obtained in the last 12 months, but the radiologist identified OA on the final report. Finally, orders were classified as “possibly appropriate” if prior weight-bearing x-rays had been obtained in the last 12 months and the radiologist had not identified OA in the final report. X-ray reports were reviewed in the patient's local medical chart and in Vista Web, which allows access to reports and most images from any VA in the country, as well as access to limited reports and images from outside facilities. Weight-bearing status of x-rays was determined by reviewing x-ray reports for the words “standing” or “weight-bearing.” If the final radiology report did not specifically note weight-bearing status, available images were reviewed for markers indicating weight-bearing status. If weight-bearing status was not reported and no images were available for review, the image was considered to be non–weight-bearing. Magnetic resonance imaging orders that were placed but not completed were included in the review. If orders were administratively canceled by radiology because no prior x-rays had been completed, these were categorized as inappropriate.

F1
FIGURE 1:
Classification scheme for knee MRI orders placed by PCPs.

Data Analysis

There were 2 observation periods for each provider: 1 year before the training date (“precourse”) and 1 year after the training date (“postcourse”). For each provider, the number of MRI orders in each category (“inappropriate,” “probably inappropriate,” “possibly appropriate”) was tallied for the precourse and postcourse observation periods. Differences in the total numbers of precourse and postcourse MRI orders for each provider were compared, as well as differences within each of the 3 appropriateness categories. Precourse and postcourse differences were evaluated using paired Student t test (2-tailed).

This project was reviewed by the institutional review board of the University of Utah and the SLC Veterans Affairs Medical Center and was determined to be exempt from institutional review board review because the work did not meet the definition of research with human subjects and was considered a quality-improvement study.

RESULTS

Twenty-six PCPs (11 physicians, 9 advanced practice nurses, and 6 physician assistants) participated in the course. As shown in Figure 2, the total number of MRI orders decreased from 130 (precourse) to 93 (postcourse), a reduction of 28% (p = 0.04). This decrease was due entirely to reductions in orders categorized as “inappropriate” or “probably inappropriate.” Differences of similar magnitude were also observed within the appropriateness categories but were not statistically significant as the power of the analysis was constrained by the sample size in each category.

F2
FIGURE 2:
Number of MRI orders by type 1 year precourse versus 1 year postcourse.

The frequency distribution of the data for each PCP is shown in Figure 3. Three providers did not order any knee MRI scans over the study period; 14 participants had a decrease in orders, 7 increased, and 2 had no change. Of note, a single provider entered a substantial proportion of the orders in both observation periods (29 [22%] precourse, 31 [33%] postcourse), represented by bar 26 in Figure 3.

F3
FIGURE 3:
Number of MRI orders by provider 1 year precourse versus 1 year postcourse.

DISCUSSION

Participation in the SLC MSK Mini-Residency was associated with a significant reduction in unnecessary orders for knee MRI scans. To our knowledge, this is one of the first analyses to show an improvement in knee MRI utilization following an accredited CPD course.

Our study had several strengths. First, our findings build on the foundation of a previously described program, which was developed through a systematic process informed by cost-effectiveness projections.22 Second, the individual components of this educational experience have become well-established, and validity evidence for the assessment measures has been critically reviewed and published.27 The findings we have reported in this article provide new validity evidence of consequences, which can inform ongoing program evaluation and course development.29–32 Finally, the enclosed nature of the VA health care system at the time of our study, including a common electronic medical record, facilitated a reliable longitudinal observation of participants' clinical behavior.

We also recognize several factors that may limit the strength of our findings and should be considered in the interpretation of our results. Most importantly, we acknowledge that we were not present when the decision was made to order an MRI, and it is possible that the nuance and complexity of a clinical encounter might provide some justification for this choice of imaging that our scheme classified as inappropriate or probably inappropriate. Second, although our framework drew from the widely recognized ACR Appropriateness Criteria, these criteria have been criticized by some clinicians, prompting other investigators to use proprietary, unpublished protocols to determine appropriateness of radiographic imaging.33,34 Third, the MRI orders we identified were only those within the VA system. It is possible that patients received care outside the VA—which may have included radiographic imaging—which we did not capture. If this occurred at all, we anticipate that these numbers would be small and would influence the pretraining and posttraining observations equally. Finally, we do not know that the PCPs' clinical duties and practice settings were equivalent in the pretraining and posttraining periods. However, many PCPs sought this training in order to develop new MSK-focused clinics in their practice locations. In this case, we would anticipate that the posttraining period would offer more opportunities for PCPs to consider ordering an MRI for patients with knee pain, which would actually increase the significance of our findings.

The findings of our study raise some interesting questions. Although this program was successful, the problem of overutilization was not completely eliminated—if the program was truly effective, why were there any inappropriate orders for MRI after the training? Also, there were several providers who actually ordered more MRI scans after training; it is possible that these participants were those associated with newly developed MSK-focused clinics, but that is not known. Perhaps the most interesting question relates to the individual participant with the greatest number of orders—what accounts for this observation in the context of a program that was otherwise largely successful?

The theoretical framework of self-regulated learning (SRL) may offer a structure with which to consider these questions and provide guidance in developing and sustaining educational programs inspired by the SLC MSK Mini-Residency. In providing one of the early perspectives to consider SRL in HPE, Durning et al.35 observed the value of this framework in helping individual learners improve performance. Although SRL includes a range of specific theories, 4 themes of learner-centered processes can generally be found across all of them: feedback (to the learner), motivation (of the learner), goals (set by the learner), and self-monitoring (by the learner).36 These themes are integrated by Zimmerman's37 3-phase model, which describes the temporal dynamics of these processes as forethought, performance, and self-reflection, occurring before, during, and after a discrete learning activity.

Using this model, the work of learning can be explored through SRL microanalysis, a technique in which a structured interview is used to examine an individual learner's clinical reasoning in real time.36,38–40 In HPE, SRL microanalysis has been pioneered in undergraduate and graduate medical education, and these results suggest that this approach may be valuable in CPD. For example, the OSCE stations in the MSK Mini-Residency could be modified to include a formal microanalytic interview, with questions posed as the learner begins the case (forethought phase), as he/she works through the station, formulating initial interpretations and management plans (performance phase), and as he/she debriefs the experience with the faculty after the OSCE has been completed (self-reflection phase).

Finally, a social-constructivist model of SRL suggests that to evaluate the effectiveness of a particular learning experience, we must have some understanding of the personal factors that an individual brings to the learning environment (i.e., their existing knowledge, skills, attitudes, and values) and the workplace factors that may facilitate or impede application of new knowledge.41,42 Course development informed by this perspective would place less emphasis on methods of instruction and assessment (with the exception of microanalysis, described previously) and more on efforts to better understand the learner—self-assessments of their current practice patterns and educational needs, their individual goals and beliefs about learning, and the details of their day-to-day experience in their workplace.

CONCLUSION

The SLC MSK Mini-Residency is a successful educational intervention that leads to decreased numbers of inappropriate knee MRI orders. This finding indicates that earlier projections underestimated the cost-effectiveness of the program, because MRI orders were not included in the prediction model. Extension of this program to other sites would help improve resource utilization, and course development informed by SRL theory could enhance the effectiveness of this experience for more learners.

ACKNOWLEDGMENT

The authors thank Matthew R. Call, DO (Utah Valley Rheumatology, Springville, UT), who prepared a preliminary version of this work, which he presented as an abstract at the 2015 ACR/ARHP Annual Meeting in San Francisco, CA.

REFERENCES

1. Jacobs JJ, editor. The Burden of Musculoskeletal Diseases in the United States: Prevalence, Societal and Economic Cost. Rosemont, IL: American Academy of Orthopaedic Surgeons; 2008.
2. Dillon CF, Rasch EK, Gu Q, et al. Prevalence of knee osteoarthritis in the United States: arthritis data from the third National Health and Nutrition Examination Survey 1991–94. J Rheumatol. 2006;33:2271–2279.
3. Nguyen US, Zhang Y, Zhu Y, et al. Increasing prevalence of knee pain and symptomatic knee osteoarthritis: survey and cohort data. Ann Intern Med. 2011;155:725–732.
4. FitzGerald JD, Battistone M, Brown CR Jr., et al; American College of Rheumatology Committee on Rheumatology Training and Workforce Issues. Regional distribution of adult rheumatologists. Arthritis Rheum. 2013;65:3017–3025.
5. Day CS, Yeh AC. Evidence of educational inadequacies in region-specific musculoskeletal medicine. Clin Orthop Relat Res. 2008;466:2542–2547.
6. Glazier RH, Dalby DM, Badley EM, et al. Determinants of physician confidence in the primary care management of musculoskeletal disorders. J Rheumatol. 1996;23:351–356.
7. Haywood BL, Porter SL, Grana WA. Assessment of musculoskeletal knowledge in primary care residents. Am J Orthop (Belle Mead NJ). 2006;35:273–275.
8. Houston TK, Connors RL, Cutler N, et al. A primary care musculoskeletal clinic for residents: success and sustainability. J Gen Intern Med. 2004;19(5 Pt 2):524–529.
9. Monrad SU, Zeller JL, Craig CL, et al. Musculoskeletal education in US medical schools: lessons from the past and suggestions for the future. Curr Rev Musculoskelet Med. 2011;4:91–98.
10. O'Dunn-Orto A, Hartling L, Campbell S, et al. Teaching musculoskeletal clinical skills to medical trainees and physicians: a best evidence in medical education systematic review of strategies and their effectiveness: BEME guide no. 18. Med Teach. 2012;34:93–102.
11. Smith CC, Newman L, Davis RB, et al. A comprehensive new curriculum to teach and assess resident knowledge and diagnostic evaluation of musculoskeletal complaints. Med Teach. 2005;27:553–558.
12. Wilcox T, Oyler J, Harada C, et al. Musculoskeletal exam and joint injection training for internal medicine residents. J Gen Intern Med. 2006;21:521–523.
13. Petron DJ, Greis PE, Aoki SK, et al. Use of knee magnetic resonance imaging by primary care physicians in patients aged 40 years and older. Sports Health. 2010;2:385–390.
14. Roberts TT, Singer N, Hushmendy S, et al. MRI for the evaluation of knee pain: comparison of ordering practices of primary care physicians and orthopaedic surgeons. J Bone Joint Surg Am. 2015;97:709–714.
15. Wylie JD, Crim JR, Working ZM, et al. Physician provider type influences utilization and diagnostic utility of magnetic resonance imaging of the knee. J Bone Joint Surg Am. 2015;97:56–62.
16. George E, Tsipas S, Wozniak G, et al. MRI of the knee and shoulder performed before radiography. J Am Coll Radiol. 2014;11:1053–1058.
17. Akesson K, Dreinhofer KE, Woolf AD. Improved education in musculoskeletal conditions is necessary for all doctors. Bull World Health Organ. 2003;81:677–683.
18. Comer GC, Liang E, Bishop JA. Lack of proficiency in musculoskeletal medicine among emergency medicine physicians. J Orthop Trauma. 2014;28:e85–e87.
19. Murphy RF, LaPorte DM, Wadey VM; American Academy of Orthopaedic Surgeons Orthopaedic Education Study Group. Musculoskeletal education in medical school: deficits in knowledge and strategies for improvement. J Bone Joint Surg Am. 2014;96:2009–2014.
20. Harkey P, Duszak R Jr., Gyftopoulos S, et al. Who refers musculoskeletal extremity imaging examinations to radiologists?AJR Am J Roentgenol. 2018;210:834–841.
21. Battistone MJ, Barker AM, Lawrence P, et al. Mini-residency in musculoskeletal care: an interprofessional, mixed-methods educational initiative for primary care providers. Arthritis Care Res (Hoboken). 2016;68:275–279.
22. Nelson RE, Battistone MJ, Ashworth WD, et al. Cost effectiveness of training rural providers to perform joint injections. Arthritis Care Res (Hoboken). 2014;66:559–566.
23. Katz JN, Shrestha S, Losina E, et al. Five-year outcome of operative and nonoperative management of meniscal tear in persons older than forty-five years. Arthritis Rheumatol. 2020;72:273–281.
24. Englund M, Guermazi A, Gale D, et al. Incidental meniscal findings on knee MRI in middle-aged and elderly persons. N Engl J Med. 2008;359:1108–1115.
25. Katz JN, Brophy RH, Chaisson CE, et al. Surgery versus physical therapy for a meniscal tear and osteoarthritis. N Engl J Med. 2013;368:1675–1684.
26. Hewson MG, Little ML. Giving feedback in medical education: verification of recommended techniques. J Gen Intern Med. 1998;13:111–116.
27. Battistone MJ, Barker AM, Beck JP, et al. Validity evidence for two objective structured clinical examination stations to evaluate core skills of the shoulder and knee assessment. BMC Med Educ. 2017;17:13.
28. Fox MG, Chang EY, et al; Expert Panel on Musculoskeletal Imaging. ACR Appropriateness Criteria® chronic knee pain. J Am Coll Radiol. 2018;15(11S):S302–S312.
29. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med. 2006;119:166.e7–166.e16.
30. Downing SM. Validity: on meaningful interpretation of assessment data. Med Educ. 2003;37:830–837.
31. Messick S. Validity. In: Linn RL, editor. Educational Measurement. 3rd ed. New York: American Council on Education: Macmillan Pub. Co.: London; 1989. p. 13–103.
32. Moore DE Jr., Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009;29:1–15.
33. Bautista AB, Burgos A, Nickel BJ, et al. Do clinicians use the American College of Radiology Appropriateness Criteria in the management of their patients?AJR Am J Roentgenol. 2009;192:1581–1585.
34. Lehnert BE, Bree RL. Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support?J Am Coll Radiol. 2010;7:192–197.
35. Durning SJ, Cleary TJ, Sandars J, et al. Perspective: viewing “strugglers” through a different lens: how a self-regulated learning perspective can help medical educators with assessment and remediation. Acad Med. 2011;86:488–495.
36. Artino AR, Brydges R, Gruppen LD. Self-regulated learning in healthcare profession education: theoretical perspectives and research methods. In: Cleland JA, Durning SJ, editors. Researching Medical Education. Chichester, West Sussex, UK: John Wiley & Sons, Ltd.; 2015.
37. Zimmerman BJ. Attaining self-regulation: a social-cognitive perspective. In: Boekaerts M, Pintrich P, Zeidner M, editors. Handbook of Self-regulation. San Diego, CA: Academic Press; 2000. p. 13–39.
38. Cleary TJ. Emergence of self-regulated learning microanalysis: historical overview, essential features, and implications for research and practice. In: Handbook of Self-regulation of Learning and Performance. New York: Taylor & Francis; 2011.
39. Artino AR Jr., Cleary TJ, Dong T, et al. Exploring clinical reasoning in novices: a self-regulated learning microanalytic assessment approach. Med Educ. 2014;48:280–291.
40. Cleary TJ, Durning SJ, Artino AR Jr. Microanalytic assessment of self-regulated learning during clinical reasoning tasks: recent developments and next steps. Acad Med. 2016;91:1516–1521.
41. Butler DL, Cartier SC. Promoting effective task interpretation as an important work habit: a key to successful teaching and learning. Teach Coll Rec. 2004;106:1729–1758.
42. Butler DL, Schnellert L. Collaborative inquiry in teacher professional development. Teach Teach Educ. 2012;28:1206–1220.
Keywords:

continuing professional development; magnetic resonance imaging; medical education; musculoskeletal

Supplemental Digital Content

Written work prepared by employees of the Federal Government as part of their official duties is, under the U.S. Copyright Act, a “work of the United States Government” for which copyright protection under Title 17 of the United States Code is not available. As such, copyright does not extend to the contributions of employees of the Federal Government.