Jamil, Hikmet MD, PhD; Campbell-Voytal, Kimberly PhD; Arnetz, Judith E. PhD, MPH, PT
Despite the fact that the numbers of US employees and worksites have nearly doubled in the past 30 years, suggesting a need for increased occupational health expertise, occupational and environmental medicine (OEM) remains one of the smallest of all medical specialties.1 Today, there are only about 3600 physicians board certified in occupational medicine in the United States.2 Moreover, there are currently only 28 formal residency training programs in OEM, compared with 36 programs in 20033 and 40 in 1995.1 Thus, the lack of physicians with specialist training, compounded by changes in the work environment, the workforce, work organization, and the health care system, have created the challenge of providing quality occupational and environmental health services.4–6
In 2007 alone, there were more than 4 million nonfatal occupational illnesses and injuries recorded in the United States.7 The scarcity of OEM physicians means that patients with these health issues often present to primary care physicians.2 Nevertheless, training in OEM is not standardized in US residency programs for family medicine physicians, and it is not known how many of them offer formal OEM rotations. A survey of 290 family medicine residency program directors revealed that, although 92% believed there was a need for training in OEM for family medicine residents, such training was only offered by 68% of respondents.2 When available, specific training in OEM only comprised at best ∼2% of the overall curriculum time. Moreover, few primary care physicians have received any earlier OEM training, because such programs in undergraduate medical education are limited.8–11 These findings are in contrast to recommendations by the Institute of Medicine, American College of Physicians, and Accreditation Council for Graduate Medical Education that primary care physicians should be able to recognize and manage occupational and environmental health hazards and conditions.2
In response to the need for physician training in OEM, one large, Midwestern medical school has offered a 4-week rotation for family medicine residents since 1994. The purpose of the course is to enhance the ability of family medicine residents to recognize, diagnose, manage, and prevent health disorders commonly encountered in the primary care setting and those that arise from, or are aggravated by, the workplace or the environment or both. The rotation also aims to familiarize residents with specific public health, legal, and governmental issues related to occupational health and safety. The program combines classroom lectures with workplace site visits and clinical placements and is held for 3 full days per week during the 4-week period. For each course, 55 to 60 hours are allotted to didactic lectures; 9 to 12 hours are devoted to 3 to 4 site visits; and 16 hours are devoted to 4 clinic visits. Participants take a pre- and posttest of their knowledge in OEM on the first and last days, respectively, of the rotation. At the conclusion of each rotation, participants are asked to evaluate the course via a short paper survey. The survey has evolved over time, but the same form has been used since 2002. Over the years, these evaluations have been useful to the rotation director in developing the program to better fit the needs and requests of the residents. Nevertheless, no formal analysis of the evaluations has been conducted. In particular, student responses to open-ended questions in the evaluation forms have never been examined in any systematic manner. These forms provide a rich data set concerning family medicine residents' perceptions of the usefulness of the didactic and clinical training in OEM. The aims of this study were to 1) evaluate residents' perceptions of the usefulness and value of training in OEM; and 2) to assess changes in residents' knowledge as measured by pre- and posttests scores.
MATERIALS AND METHODS
The rotation was offered 4 times per year, with 6 to 12 participants per rotation. The current project entails a qualitative analysis of OEM rotation evaluations collected from family medicine residents between 2002 and 2008. In addition, resident's knowledge of key subject areas in OEM was measured at the beginning and conclusion of each rotation. Changes in resident knowledge of OEM measured by the pre- and posttests were triangulated with residents' perceptions of the usefulness of the rotation. Ethical approval for this study was granted by the Human Investigation Committee, Wayne State University.
Evaluation forms were distributed during class time in the last week of the rotation, without the rotation director present. All forms were completed anonymously and included no personal identifiers. Students who wished could decline the opportunity to complete the evaluation without penalty by returning the form unanswered. Residents responded to five open-ended questions regarding the 1) usefulness of the rotation postresidency; 2) aspects of the rotation that were considered most useful; 3) new skills that were acquired; 4) new information that came as a surprise or altered one's view of occupational health; and 5) perceptions of the importance of incorporating an occupational history into a patient's general medical history. Once completed, evaluation forms were placed into an envelope, submitted to the division secretary, and returned to the rotation director at the completion of the rotation.
Pretest and Posttest
Residents were asked to fill out identical pre- and posttests of their knowledge of OEM at the beginning and conclusion of each rotation. All tests were filled out anonymously, with no information regarding age or gender. However, students were asked to provide the hospital residency program they were attending and the last four digits of their social security number, so that individual pre- and posttests scores could be linked. The questions in these tests were derived from questions included in Family Medicine board certification examinations and from a pool of 1000 questions that had been used in OEM residency programs.
Participants were family medicine residents attending the 4-week OEM rotation between 2002 and 2008. Residents were from one of six local hospitals, representing different family medicine residency programs. The OEM rotation was mandatory for the residents of two of the six programs and elective for the other four. Course attendees were either second- or third-year residents. More than 50% of course participants were women, and the majority of them were foreign medical graduates. A total of 254 students attended the rotation in the 7-year period. This analysis focused only on rotation participants who were family medicine residents. All medical students, fellows, and residents in other specialties (n = 24 participants) were excluded. The final study population was thus comprised 230 family medicine residents.
Procedures and Data Analysis
Evaluation forms were organized by year and month and assigned an identification number. A grounded theory12 approach was used for the qualitative analysis of the five open-ended questions. The analysis was completed without any preconceived ideas, thoughts, or theories regarding findings. Two of the researchers independently completed an initial review and open-coded a subset of the course evaluations. Impressions and codes were then reconciled, and an initial coding tree was developed. This coding schema was subsequently applied to a subset of data. Through iterative discussions, the code tree was revised, and operational definitions for each code were sharpened. Once a final code tree was confirmed, all data were recoded by the same two researchers. Interrater reliability of at least 80% was achieved on a random sample of evaluation forms drawn from each year (n = 14). Once coding was complete, the data were organized into a preliminary set of themes, which were then shared with the rotation director. The director, who was not involved in the coding, offered an important historical perspective on the data set that increased the validity of the interpretations.
For the purpose of analyzing the pre- and posttests, the 40 items in the test were grouped into 4 subject categories: chemical (6 items); health and disability management (4 items); work and environment (10 items); and clinical (20 items). Questions in the chemical category dealt with radon, poisoning, and various chemical exposures associated with occupational disease. The health and disability management questions concerned workers' compensation and return-to-work issues. Examples of work and environment questions included items on noise-induced hearing loss, industrial hygiene, and legislation regarding occupational health. Clinical items concerned recognition of clinical findings for a range of specific occupational illnesses. Each test item was scored 0 for an incorrect response and 100 for a correct response. The total score for each category was divided by the number of items in that category, so that the maximum possible score for each category was 100%. Mean scores were calculated for individual responses on all questions combined and for each subject category. Pre-and posttest scores for all respondents were compared using paired samples t tests for total scores, subject scores, and comparisons within groups (years and residency programs). When comparing groups with small numbers, Wilcoxon signed rank tests were also used, yielding identical results. To see whether changes in knowledge differed by year and by residency program, we used one-way analysis of variance (ANOVA) with post hoc tests. ANOVA was run on delta values for pre- and posttests scores for both total scores and for individual subject scores. Significance for all analyses was set at P < 0.05 (two tailed). The SPSS statistical software package version 16.0 (SPSS, Inc, Chicago, IL) was used for all analyses.
Rotation Evaluations: Qualitative Findings
A total of 208 evaluation forms were completed during 7 years. This represents responses from 90% of the 230 family medicine residents enrolled in the 28 rotations. Because surveys were anonymous, data on the age and gender distribution of the respondents are not available. Responses to the open-ended questions were grouped into three distinct categories: knowledge, experience, and skill development. A summary of the code definitions and key subcodes that emerged within each category is presented in Table 1.
Knowledge: Content-Specific Learning
In their appraisal of the rotation, residents reported improvement in their knowledge of OEM both generally and in specific areas. Many commented on gaining a general awareness and insight as to the relevance of occupation to patient health.
“This rotation will be very useful. A large portion of patients who come in to my clinic have jobs and spend 40+ hours a week working. This rotation really made me realize what a significant impact occupation has on one's health, from sprains, strains, and lacerations to more chronic diseases.”
Another commented that:
“[It has] definitely increased my awareness of how work relates to illness. We spend more time at work than anywhere else and the fact is that most injury is preventable. Education of patients and MDs can have an impact on prevention.”
Residents also felt they had gained more in-depth knowledge of occupational medicine, including specific exposures, work-related and environmental hazards, injuries, illnesses, and return-to-work issues. Several mentioned that they would be able to use this new knowledge directly, such as in identifying work-related problems, workplace stressors, and educating patients about toxic substances. Residents expressed the impact of this new knowledge on their current work as family medicine physicians:
“I am now more confident in dealing with work related issues. Surprisingly, while in this rotation four of my patients came to me with such issues. Now I ‘hear’ more.”
“I learned things which I hadn't learned in the last 3 years, the things unique to occupational medicine but part of family practice.”
Another subgroup under the overall theme of new knowledge concerned legal information. Residents reported having become more familiar with policies regarding workman's compensation benefits, disability, family medical leave, and recordable Occupational Safety and Health Administration (OSHA) conditions. Residents also became more aware of resources for tracking and updating their knowledge.
“I was not aware of the details of the ADA [Americans with Disabilities Act], and also issues about confidentiality.”
“[I] got a good understanding of worker's compensation, assessment of work related injuries, assessing the return to work time period, work restrictions etc.”
Finally, residents reported gaining new practical knowledge of occupational medicine clinics and how they work, and of the resources that are available for employees/patients and physicians.
“I really did not understand what an occupational physician did. Now I have a better understanding how OSHA, NIOSH [National Institute for Occupational Safety and Health], EPA [Environmental Protection Agency], industrial hygienists, occupational medicine physicians all contribute to patients'/employees' health on different levels.”
The second major category of residents' comments concerned the experiential value of viewing conditions in factories and rotating through occupational medicine clinics. In general, residents were surprised about the “reality” of work and how the work environment (factory, shop floor) relates to health and ultimately to primary care practice. For many of the residents, the rotation gave them their first exposure to different types of industrial workplaces. Several comments expressed appreciation for being able to observe these real-life situations. One resident stated:
“The site visits, especially at car manufacturers, allowed me to see what tasks my patients actually do so I better understand how to evaluate their work-related injuries.”
“The worksite visits were most surprising to me, I gained a firsthand look at how people work” and “... understanding a variety of job duties within a given job title. [These gave] direct visualization of certain jobs, ie, line work.”
“I was surprised at what some employees are subjected to on their job.”
Visits to occupational health clinics were also appreciated. The clinics also gave them a better understanding of the multidisciplinary nature of occupational medicine, and they recognized the value in conducting an occupational history in primary care.
“Clinical exposure to occupational medicine (ergonomics, engineering control) [was most useful]. “[This]…was extremely helpful in helping me develop a sense of how to conduct an occupational history and its importance in primary care.”
The third major category of comments concerned skills that the residents had developed during the rotation. Skills development was reported in three areas: 1) diagnostic skill, 2) documentation skill, and 3) specific diagnostic techniques. By far, the most frequently mentioned skill was the ability to conduct a comprehensive occupational history and physical. This was expressed as developing an occupational perspective and incorporating that into their medical practice.
“I can recognize and evaluate occupational related and environmental health hazards and appropriately take actions or refer.”
The value of conducting a detailed occupational history and physical was generally recognized.
“I have always asked for occupation [in the past], however, I never asked in more detail to find out what they actually do all day at work. I didn't always ask if they sit or stand when they work, or if they lift and what type of lifting is involved. I realize how easily a diagnosis could be missed if the work history is not collected in detail.”
“It is very important in all aspects of disease, in fact, it seems impossible now to make an objective evaluation without [conducting an occupational history]. The differential diagnosis would be incomplete. I wish I took this course much earlier.”
In addition, residents recognized the importance of considering the interaction between workers and their work environments in increasing the precision of the physical assessment. Participants mentioned this in regard to specific work-related health risks, such as noise, toxic exposures, and the potential for musculoskeletal injury. Patients seeking help for pain relief were also viewed in a somewhat different light for many of the residents attending the rotation:
“Some patients were considered pain [medication] seekers by their primary care physicians, but it can be work related.”
Residents also mentioned specific diagnostic skills such as conducting a Department of Transportation physical examination, a focused assessment of work-related injury, and the ability to determine when an employee is ready to return to work.
The second most frequently identified skill area was associated with completing regulatory forms and following legal policy and regulations common for work-related injuries and illnesses. Occupational health policies and work regulations are complex, and residents reported feeling more confident in documenting injuries and setting work restrictions.
“[I learned]... how to pursue a case as to whether it is reportable or not, and when to give leave from work versus restrictions.”
Finally, residents reported the development of new technical skills, such as the use of the slit lamp, techniques for removal of a foreign body from the eye, advanced assessment of musculoskeletal injury involving joints, hand, and spine, and finally, specific techniques for assessing pain. Comments included:
“I learned how to perform a great musculoskeletal examination!” and “I have further tuned my skills for diagnosing back pain.”
Pretest and Posttests: Quantitative Results
Pre- and posttest scores were available for 217 of the 230 respondents (94%). A comparison of total pre- and posttests scores for all residents revealed a significant improvement in resident knowledge, from a pretest mean of 42.4% to a posttest mean of 57.2% (mean difference = 14.83, 95% CI = 16.4 to −13.3, P < 0.0001). Total scores did not differ significantly by year (F = 1.70, df = 6, P = 0.12) or by residency program (F = 1.05, df = 5, P = 0.39). However, improvements within each year and within five of the six residency programs were significant (P < 0.001). There was a significant improvement in resident knowledge in each of the four subject categories (P < 0.001, Fig. 1), with significant differences for the chemical area between 2002 (mean = 95.8, SD = 126.8, 95% CI = 42.3 to 149.4) and 2007 (mean = 246.4. SD = 155.1, 95% CI = 186.3 to 306.6, P < 0.01). Otherwise, the improvements in subject area knowledge were consistent over time.
Residents' perceptions of the usefulness and value of the OEM rotation as expressed in their evaluation comments reflected an improved ability to recognize, diagnose, and manage work- and environmentally related health conditions that may be encountered in a primary care setting. Furthermore, many participants reported gaining insight regarding the possibilities for injury and illness prevention, using the newly acquired knowledge and skills provided by the rotation. By gaining an increased awareness of the significant role of occupation and environment in one's health, the residents came away feeling better able to perform their duties as family physicians. Comments were positive and stable over time, reflecting no major deviation in residents' appraisal of the rotation from year to year. These subjective perceptions were supported by significantly improved knowledge scores in all groups over time. Mean values for delta scores in the chemical subject area were significantly lower in 2002 compared with 2007, which might be explained by the fact that no toxicology lectures were offered in 2002. Otherwise, improvement in knowledge scores was consistent across years.
In similar work, Grime et al10,11 studied perceived value of a half-day teaching session in occupational medicine among undergraduate medical students in the United Kingdom. They reported high ratings for usefulness and relevance, as well as an appreciation of case histories, workplace visits, and occupational history. However, those reports concerned a single teaching session for undergraduate medical students, not residents seeing patients on a regular basis.
Many residents commented on the benefit of the worksite visits, which gave them a chance to visualize people performing different work tasks. Several stated that this was the first time they had seen such workplaces, including factory floors, industrial plants, and assembly lines. Cordes et al13 asked medical students to evaluate a 5-week summer course that incorporated worksite visits into the clinical medicine curriculum. Students were only slightly to moderately satisfied with the educational value of the site visits, depending on the particular site. Residents in this study commented on the benefit of the rotation design, where classroom didactics provided a foundation for much of what was observed and experienced at worksites and occupational health clinics. At the same time, the rotation was held only 3 days per week, giving residents the opportunity to apply their newly acquired knowledge and skills in their regular patient contacts in the remaining 2 days. The notion of “seeing” and “hearing” work-related issues for the first time in their interactions with patients was reflected in many of the residents' evaluation comments.
The importance of taking an occupational history was universally emphasized by the residents. Many pointed out that it enhanced their ability to make a proper diagnosis and to devise suitable treatment. In addition, many of the residents understood that a proper occupational history entailed more than simply asking the patient what they did for a living, but rather, investigating in detail what the job entailed. Cimrin et al14 reported that less than 23% of 66 interviewed physicians took a detailed occupational history from all of their patients. The authors attributed this to a lack of training and understanding of its importance.
As in all qualitative studies, the subjects-reported perceptions and experiences provide evidence that is not necessarily generalizable to all family medicine residents. Qualitative results were based on responses from 90% of the 230 family medicine residents enrolled over the 7-year period, reflecting an excellent response rate. However, it is not known whether the residents in this study, many of whom were foreign graduates, were representative of family medicine residents nationwide. Research indicates that the proportion of international medical graduates in family medicine has increased in recent years in the United States.15 Nevertheless, little is known about nonrespondents or whether their evaluations would have differed significantly from those of the residents who did respond. However, we believe that selection bias is minimal, considering the consistency of responses over the 7-year period. This same consistency leads us to believe that our study has captured important aspects regarding perceived usefulness and value of training in OEM.
The OEM rotation was mandatory for residents from two of the programs and elective for the other four. This was another possible source of bias, in that the majority of students had chosen to study OEM, which might reflect an existing interest in the subject. However, evaluation form comments were almost consistently positive, and we do not believe that the students for whom participation was mandatory were significantly less positive. Moreover, we found no statistically significant differences in improved knowledge between the different residency programs.
Finally, this study covers a long period of time, during which there had been some variation in course content, site, and clinic visits, and individual lecturers and preceptors. Thus, despite the consistency of both resident evaluations and knowledge scores over time, it should be noted that experiences and course content, though similar, were not identical across rotations.
The enthusiasm and positive evaluations by the family medicine residents in this study suggest that a rotation in OEM such as ours could be of relevance for other family medicine residents and physicians. Moreover, increased knowledge of the role and significance of occupational and environmental health may be of value to physicians within other specialties. Future research should explore the perceived benefits and value of such training in other physician groups. In addition, a follow-up survey of residents who attended the 1-month course will explore the long-term perceived value of the OEM training on their family medicine practice.
The authors would like to thank all of the lecturers and worksite and clinical preceptors who have participated in the OEM rotation over the years.