The Patient-Centered Medical Home (PCMH) provides “accessible, continuous, comprehensive, and coordinated care that is delivered in the context of family and community and this model of care is encouraged by most primary care associations.”1,2 Under the PCMH model, care must be provided for a wide variety of complaints and coordinated among many specialties. Health care that is comprehensive and coordinated, then, requires consideration of the patient's job, as working patients may spend close to half their waking hours at work.3 Increased integration of factors relating work and health into primary care could contribute to the success of the PCMH model. Research indicates that primary care physicians provide care for patients who have a high frequency of concerns related to their job.4 However, research also indicates that over 70% of physicians fail to record occupational information.5
The implementation of an electronic health record (EHR) system can support and align with the requirements for establishing a PCMH6,7 and meaningful use regulations for EHR implementation requires the use of CDS (Available at: https://www.healthit.gov/providers-professionals/achieve-meaningful-use/core-measures-2/clinical-decision-support-rule). CDS supplies clinicians, staff, patients, or other individuals with clinically relevant knowledge to enhance health and health care.8 Effective clinical decision approaches have been shown to change clinician behaviors9 and could be used to improve the collection and use of patients’ work information. CDS can be used to deliver information at the time of care to educate patients and clinicians about risk factors encountered at work, provide diagnostic advice regarding occupational diseases and injuries during patient assessment, and supply guidance to inform return-to-work (RTW) decisions regardless of etiology. They can reinforce prevention efforts and promote better care of working patients.10
In 2014, the National Institute for Occupational Safety and Health (NIOSH) initiated a research study to develop and evaluate the perceived value and feasibility of CDS addressing the relationship between work and health targeting PCPs. NIOSH chose as foci for the CDS clinical outcomes and information that are commonly encountered by general PCPs that involve different aspects of the interface between work and health, including 1) diagnosis and management of an occupational disease (work-related asthma), 2) consideration of work environmental factors in managing a chronic disease (diabetes), and 3) guidance for making RTW determinations for patients when a health condition, such as low back pain, might limit their ability to perform their full work activities.
As mentioned by Filios et al,11 the first step in the broader NIOSH study was the identification of clinical guidelines and other suitable reference materials by subject matter experts (SMEs) to create a CDS Knowledge Resource for three topics described above. Knowledge Resources contain narrative information and the decision logic (flow diagrams) to explain what the CDS would do. They also include the scientific rationale and the evidence-base for the specific information or recommendations provided and explain how that information is clinically relevant. Further explanation of this process, as well as the three Knowledge Resources developed by the SMEs are presented in accompanying articles in this same journal issue.
Once an initial Knowledge Resource has been developed, it is then important to gather feedback from multiple EHR system users in different clinical settings, to ensure the logic and information provided meet their needs. Qualitative methods have become widely accepted in health informatics as an effective approach for gathering this kind of feedback and providing formative assessments so that changes can be based on scientific evaluation. In this article, the authors provide the results of the qualitative assessment of the Knowledge Resources through systematic collection of feedback from multidisciplinary clinicians and others associated with five diverse primary care networks.
International guidelines about the conduct and reporting of evaluation studies in informatics focus as much on qualitative methods as they do on quantitative. For this study, we followed established qualitative approaches recommended by Nykanen et al12 in their Guideline for Evaluation Practice in Health Informatics. Five sites were selected to reflect the diversity of primary care group practices from the perspective of geography (located in five different Health and Human Services regions), experience with using an EHR (three different EHR vendors and early adopters as well as recent adopters), and type of health care organization, including two federally qualified health centers (FQHCs) and three academic and community-affiliated primary care practices. Although not selected on this basis, all five sites had been certified as PCMHs.
We defined “provider” as someone who can order care (eg, physician, nurse practitioner, physician assistant, etc). We defined “clinician” to include providers and other members of the health care team, such as medical assistants (MAs) and care coordinators, who see patients. Subjects were selected with the assistance of an inside contact person based on their roles as providers, clinicians, and other health care team members, that is, information technology, quality, or informatics professionals, or as managers/administrators. We requested that our clinic contacts recruit as broad a selection of individuals as possible so that EHR skeptics, champions, and others in between were included. For those group practices with multiple practice sites, an attempt was made to choose respondents from a variety of sites.
Data were collected primarily through semi-structured interviews and observation of workflow in clinics and of the work of individual clinicians, with a focus on the role of the EHR in the clinical encounter. We used the Rapid Assessment Process13 that was adapted by health informatics researchers14,15 and it is based on a rapid ethnographic assessment methodology used by public health researchers.16 We followed guidelines for assuring rigor by triangulating (multidisciplinary researchers, multiple sites, different types of data), member checking (feedback to sites about results), auditing (tracking data gathering), practicing reflexivity (researchers’ awareness of their own bias), and reaching data saturation (gathering data until little more is being learned).
Each interviewee was scheduled for a 30-minute time slot. The interviews began with questions related to the respondents’ daily work patterns and their experience with EHRs and CDS. This was followed by the respondents’ general attitudes and practice regarding consideration of health factors encountered in a patient's job and questions about how a patient's work information is collected in the EHR. To specifically collect feedback on the three Knowledge Resources and because of our limited interview time, we presented interviewees with three scenarios that briefly summarized the main purpose for each of the three CDS Knowledge Resources (Table 1). We asked interviewees which scenario would be most useful to them and to their practice, and then to rank the three scenarios and describe to us their justifications for the rankings. We then gathered feedback on each scenario starting with their top choice and continuing to other scenarios depending on the interviewees’ level of interest and time constraints. As part of this feedback, we provided additional content information drawn from the full Knowledge Resource documents including the Knowledge Resource decision logic flow charts17 and specific examples of the educational information that would be provided to clinicians and patients. The full Knowledge Resource for each topic was also available and was provided to interviewees’ depending on level of interest.17
To verify what we learned through interviews, and to gather more data about workflow, we also conducted observations of clinic personnel in all roles. We shadowed individuals as they went about their daily work. In addition, we watched the flow of activity throughout the clinics. The researchers handwrote field note jottings while on site and transformed the jottings into full field note documents soon after. These were analyzed along with the interview transcripts.
Data were analyzed with the assistance of qualitative data analysis software (NVivo 10; QSR International, Doncaster, Victoria, Australia) in three ways. First, a template method was used18,19 to identify answers to our interview questions. Second, a grounded hermeneutic approach was taken to discover patterns and themes across sites.18,19 Finally, detailed comments about the three CDS Knowledge Resources were analyzed by two investigators (SB and MF) and results were triangulated with the analysis done by a third investigator (JA) who reviewed and coded all of the transcripts. In qualitative analysis, codes are terms applied to important parts of text to describe content and are somewhat like indexing terms, but they arise from the text and are not predetermined. Examples of coding terms included “important to clinical care,” “standardizes care,” and “requires too much time.”
We obtained Institutional Review Board (IRB) approval from NIOSH and the Oregon Health and Science University (OHSU) and each local site, where required. All five study sites reviewed the study protocol. Two sites required local IRB approval, one study site IRB accepted the review of the other IRBs, and two other study sites had other review procedures that were followed.
We interviewed 76 individuals across the five primary care sites, with a range of 10 to 22 individuals in any one site. Most (N = 61, 80%) interviewees were clinicians, including 38 (50%) providers, of whom most 32 (84%) were physicians. Fifteen (20%) of interviewees were clinical informaticians and health IT personnel, and seven (9%) were various other management and clinical support staff. IT personnel across all of the sites reported they had the technical capability to implement the CDS described in the Knowledge Resources. All of the EHR vendor products except one provided tools that local developers could use to build site-specific CDS incorporating the information provided in the Knowledge Resources. This finding and other issues related to the technical feasibility of implementation of the Knowledge Resources have been reported previously.20
We also conducted a total of 30 hours of observation (range: 3 to 10 hours per site) of clinicians using the EHR during clinical encounters, of other care team members, and of clinic workflow. Our observations supported our interview findings; most clinicians were not accessing any occupational data related to work exposures in the EHR and any health issues encountered at work that did arise were handled inconsistently.
In general, when selecting which scenario to discuss first, individuals told us that their choice was sometimes based on the need within their patient population, sometimes on suspected ease of implementation, and sometimes on their own personal belief systems about what is important. Despite having busy clinic schedules, the respondents across all five sites often stayed longer than scheduled, and many provided feedback on more than one scenario and their corresponding Knowledge Resource. Across all of the practice sites, respondents had very favorable reactions to all three scenarios and generally expressed reluctance to rank just one because as one commented, “I think all three of them would be (useful). They’re certainly three that we struggle with. So I mean, I could pick either one.” Forty-eight (63%) of the subjects did provide a ranking and 29 (60% of those who provided a ranking and 38% of all respondents) selected the diabetes scenario as their first choice.
Despite the diversity of practice sites, the response to the Knowledge Resources was remarkably consistent across sites. The only major difference between practice types that occurred with regularity was in response to the asthma Knowledge Resource and was a result of the lack of spirometry in the two FQHCs. This, however, was not a barrier to respondents at the FQHCs choosing and commenting on other components of the asthma Knowledge Resource. On the basis of the respondents’ comments, seven common themes emerged about the Knowledge Resources. Examples of common comments that are especially succinct for each of the Knowledge Resources for the seven themes are summarized in Table 2. Similarities and differences in the relative emphasis of these themes across Knowledge Resources are described in the following sections.
Theme #1: The Knowledge Resources Targeted Medical Outcomes that are Important to Primary Care Practices
This was the dominant theme across all the Knowledge Resources and was the most common reason given for diabetes being ranked as most important. Comments highlighted how PCPs were keenly aware of the impact of work on their patients’ health. For example, one interviewee stated: “I’m interested in the asthma because I feel like there's a lot of jobs that my patients are being put at risk for.” Not only did they see work as a risk factor for chronic diseases, but they also saw patients’ chronic disease as impeding their patients’ ability to work. For example, one respondent explained: “it's almost like what's the most important, what's the most like life-affecting…. if you don’t control your diabetes well enough then obviously you’re going to be out of work more.”
Theme #2: Providers Recognized They Need Additional Information in Order to Manage Patient's Health and Valued the Reminders the Knowledge Resources Provided
Respondents appreciated the information they would receive from the Knowledge Resources because it would improve their clinical care. This theme was especially common in response to the RTW knowledge resource. Many providers commented on how ill-prepared they felt to generate RTW letters for their patients, as one comment illustrates: “Many of us don’t really know what some of the guidelines would say. ……I sometimes make things up. ” Similarly, respondents felt that the asthma knowledge resource would provide specific information to improve their current patient assessments, and as asthma is an uncommon diagnosis in adults, a reminder was helpful: “even if it's less commonly deployed, it would still be...a good reminder to think about that with patients of asthma.”
In discussing the diabetes Knowledge Resource, respondents felt the information provided them suggestions for potentially untapped pathways to improve disease management: “I did have a diabetic patient who really did work nights. And that mattered because he was on like 70/30 insulin……we found then he was having some hypoglycemic episodes and then we realized he should be changing these things because he was up at night and eating and then sleeping during the day.” Another respondent echoed this sentiment: “one of the things that we try to do is link patients to information to help them. …with a patient who seems to have a hard time managing you know, their hypoglycemia. If I know that and it's work-related then I can refer them to the appropriate nurse to try to help them figure out better ways to manage that.”
Theme #3: The Knowledge Resources were Helpful Because They Promoted Consistency and Standardization of Care
Although respondents were well aware of the importance of work and work exposures to patient health, they were interested in having access to clear guidelines or standards of care. Although mentioned in support of all three Knowledge Resources, this was most commonly brought up in support of the RTW scenario: “we’ll tell people, yeah, they should be off work. Or they can’t lift. And when someone wants something more specific, I think we’re pulling stuff out of our pocket.”
Theme #4: Respondents Wanted More Evidence to Support how the Clinical Recommendations in the Knowledge Resources Would Improve Care
Although respondents suggested that these Knowledge Resources might contribute to standardization, they also wanted to be assured that there was a clear evidence base to justify the clinical recommendations that were provided by the Knowledge Resources.
Theme #5: Whether the CDS Tool Developed from the Knowledge Resource Takes up or Saves Time is an Important Factor in Clinician Acceptance
The amount of clinical time the CDS tools developed based on the Knowledge Resources would require was a prominent concern. For example, several suggestions for improving the diabetes Knowledge Resource concerned simplifying the number of trigger questions. One respondent pointed out that: “Each one of them seems like a simple question but that question might take five minutes of explanation of, you know, tangents, of emotion I mean, there is no more room left to talk to the patient.” For asthma, some respondents felt the trigger questions were unnecessary because they duplicated information that is already gathered: “So anybody that comes with asthma, you’re going to ask environmental related questions. I don’t usually start with work. But it would be in the field.”
On the contrary, much of the enthusiasm for the RTW Knowledge Resource was because respondents felt it was an example of how the EHR could make their work more efficient and save time: “But I’ve come along to some of the things that the EMR [electronic medical record] helps me do, like write letters to physicians and write letters to patients…It would be very helpful.” They saw benefits not just for patients but for themselves: “ there's a gain directly for health care as well as the patient, … with all of the focus on population health and reducing cost and focus on population, I think that's an area where (there is) mutual gain.” One specific mechanism for this mutual gain came from respondents’ belief that this tool could improve provider–patient interaction: “As a clinician [the back to work issue] feels like a place where we don’t get a lot of guidance. There's not a lot of resources for kind of an independent evaluation or a functional evaluation of what they are doing. And so that is always a struggle for me mentally and kind of intellectually but also as a relationship to the patient. So managing that long-term relationship, as well as making sure that you are doing an accurate assessment…there's not a lot of good resources for us to do that.”
Theme #6: The Perceived Sense of Job Insecurity Means Patients and Providers Cannot or Do Not Act on the Providers’ Recommendations and This Causes Frustration and Limits Effectiveness
Providers frequently expressed powerlessness to effect change in the working conditions that were damaging their patients’ health. Although most commonly raised by FQHC providers who see predominantly low-income patients, this theme did emerge across all practices. This challenge often discouraged providers from raising these issues: “Just identifying that there's a problem is actually not helpful. It's what the solution is to that trap. So for any one of these things, what is the expected outcome? And who has to drive that outcome? And who's the right person to drive that outcome because otherwise you just put something out there without a solution.”
The frustration providers felt in not knowing how to help their patients was expressed in many different courses of action. Some recognized that simply officially recognizing and documenting the condition was an important first step: “I’m thinking of some employees that I knew when we were still using latex gloves and they developed asthma …their only option was just to leave and find another job but this would be like documentation in their chart to help them, you know, support a job transfer or whatever to protect (them).” Other respondents expressed frustration at the ineffectiveness of their actions: “what if I wrote a letter to your boss? And I did this with someone, ‘Please let her eat?’ But it never seems to really get better.” Others felt they needed to take some action, even if it was not optimal treatment: “most people are not going to want to change their job… it's always hard to think about what you can recommend to people in terms of, like, masks. Is that enough? What I end up doing is grabbing a handful of masks and giving it to the person… I never know if it's the right mask.”
Theme #7: Another Member of the Care Team or the Patients Themselves Could Enter Some of the Information to Assist with Workflow
To save time, many respondents felt that some information could be gathered by the MAs or through patient self-administered questionnaires. For example, as many of the trigger questions were related to patient symptoms, some interviewees felt that the questions could be asked as part of a patient form that was completed before the visit because “this information is completely coming from the patient. Doesn’t matter if I ask or the patient puts it in that iPad.” Others thought the MA could initiate the process: “What brings you in? Oh, you’re having back pain? Oh. I am just going to ask you to fill out this form while the doctor is waiting to see you in a few minutes.”
However, whether the trigger questions were entered by the patient or an MA recorded the answers, most respondents felt that the provider was uniquely qualified to act on the information. “I guess this would be more a physician-patient interaction just because of the nuances in discussing the, sort of ability to breathe. There would be a lot of things to think about.” Similar sentiments were expressed related to the RTW scenario: “sometimes though, for the best intentions or not, they like to manipulate the interaction in some sort of way. It would probably, you know, the fewer involved, the better.”
Diabetes was the only Knowledge Resource where respondents felt that others in the care team could play a central role with collection of information: “I think that could certainly be something to be collected by a diabetic educator, … or nutritionist while they are developing plan of care. ….So yeah, I don’t think they would be adverse to that in the least bit. Because it wouldn’t affect them much.” Although respondents also emphasized the unique role of the providers, “I’m not sure that the MAs have enough understanding about different work and the risks of hyper or hypoglycemia to be able to figure out where the safety problems may or may not be so I think the MA could start some of the structure things but …. it would need to be a clinician.”
Our findings point to several areas for future research that could have important implications both for occupational medicine and primary care. Our study found that PCPs across a range of practice settings recognized the importance of factors encountered at work to their patients’ health. They were receptive to proposed ideas that could provide accessible and evidence-based information and tools to improve their care plans. The IT staff at these practice sites reported that the existing CDS technology within their EHR systems could feasibly incorporate the narrative information and the decision logic recommended within the Knowledge Resource reports. Those Knowledge Resources that provided assistance in managing the most common and often poorly controlled chronic health conditions (especially diabetes) and tools that improved the standardization of clinical practice for topics that were unfamiliar to most PCPs (especially RTW recommendations) garnered the greatest enthusiasm.
Another major barrier to PCPs discussing health factors encountered at work with their patients was their perceived ineffectiveness in providing information and assistance to their patients to improve working conditions. Although they recognized the importance of changing unhealthful exposures in the workplace, in the absence of clear information and evidence-based standards, they often felt powerless to recommend or successfully trigger changes in their patients’ workplaces. For some providers, these barriers led them to avoid making work-related recommendations. These findings point to the need not only for the types of occupational medicine-related Knowledge Resources reported in the accompanying articles and assessed in this study but also for other forms of collaboration between PCPs and occupational physicians such as those piloted by the Migrant Clinicians Network for farmworkers.21,22
Our study points to the important role that the EHR could play in improving occupational illness and injury surveillance systems by improving the recognition and the capture of work-related conditions treated in primary care practices.23,24 Currently, the most widely used occupational surveillance data sources likely substantially and systematically undercount certain work-related diseases and injuries, certain types of workplaces, and certain populations of workers.25–27 The EHR provides new opportunities to improve surveillance by capturing cases seen by PCPs, especially for workers reluctant to report their case to workers’ compensation or to their employer or where the relation to work was not previously recognized. Ultimately, more comprehensive surveillance systems will advance the development of better intervention and prevention strategies including those tailored to the primary care environment. Our study indicates that PCPs do in fact recognize the important role that work plays in their patients’ health and would be receptive to CDS tools that assist in their management.
Although use of the qualitative assessment approach for this study is effective for collecting a large amount of information in a short amount of time, our findings are limited based on the number and range of practice sites and clinicians we included. Those clinical sites that agreed to participate in the study may employ a select group of providers who are potentially more aware and open to consideration of work in patient care. However, most of our interviewees were not aware of the study before our arrival and were not involved in the arrangements that led to their practice's participation. A larger ethnographic study that included more sites might have provided additional information, but the commonality of findings we obtained across a range of clinical settings and types of clinicians suggested that we reached a satisfactory level of saturation. Although we only interviewed a sample of the staff in each practice site, we also conducted observations of how clinicians used the EHR during clinical encounters.
In conclusion, further developing resources such as CDS tools to assist PCPs with diagnosis, treatment, and management of health conditions in relationship to their patient's work is an important direction for occupational health research. Our study found that PCPs are aware of the importance of work to their patients’ health and are eager to gain access to information that will improve the quality and efficiency of their management of issues where work and health intersect. More systematic collection of this information from primary care practices could supplement other occupational injury and illness clinical data and ultimately improve understanding of both the impact of work on disease and injury as well as the impact of disease and injury on an individual's ability to work; it would guide and advance the development of better intervention and prevention strategies. Occupational medicine specialists play an important role in generating data to create a solid evidence base for guidelines and recommendations that will improve the treatment of injuries and illnesses at the interface of work and health by all clinicians, including PCPs. Fostering greater collaboration between PCPs, occupational health clinicians, and other community-based educational and intervention resources can support improved intervention and prevention strategies for workers.
The authors would like to thank members of the Research Team: Genevieve B. Luensman, PhD, Richard N. Shiffman, MD, MCIS, Jane F. Wiesen, PhD, James McCormack, PhD, Elizabeth V. Murphy, MD, MPH; key individuals at the five clinical sites: Rose H. Goldman, MD, MPH, and Laura Brightman, MD; Stacey Curry, MPH, and Larry J. Knight, MSA, MSHRM; Deborah Lerner, MD and Herb K. Schultz; Michael Rabovsky, MD; Joseph Conigliaro, MD, MPH, and Nicole Moodhe Donoghue.
3. US Bureau of Labor Statistics. Economic News Release. Table 1. Time Spent in Detailed Primary Activities and Percent of the Civilian Population Engaging in Each Activity, Averages per Day by Sex, 2015 Annual Averages. Washington, DC: Bureau of Labor Statistics; 2016. Available at: https://www.bls.gov/news.release/atus.t01.htm
. Accessed March 22, 2017.
4. Harber P, Mullin M, Merz B, Tarazi M. Frequency of occupational health concerns in general clinics. J Occup Environ Med
5. Politi BJ, Arena VC, Schwerha J, Sussman N. Occupational medical history taking: how are today's physicians doing? A cross-sectional investigation of the frequency of occupational history taking by physicians in a major U.S. teaching center. J Occup Environ Med
8. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc
9. Roshanov PS, You JJ, Dhaliwal J, et al. Can computerized clinical decision support systems improve practitioners’ diagnostic test ordering behavior? A decision-maker-researcher partnership systematic review. Implement Sci
11. Filios MS, Storey E, Baron S, Luensman GB, Shiffman RN. Enhancing worker health through clinical decision support (CDS): an introduction to a compilation. J Occup Environ Med
12. Nykanen P, Brender J, Talmon J, et al. Guideline for good evaluation practice in health informatics (GEP-H). Int J Med Inform
13. Beebe J. Rapid Assessment Process: An Introduction. Walnut Creek, CA: AltaMira Press; 2001.
14. Ash JS, Sittig DF, McMullen CK, Guappone K, Dykstra R, Carpenter J. A rapid assessment process for clinical informatics interventions. AMIA Annu Symp Proc
15. McMullen CK, Ash JS, Sittig DF, et al. Rapid assessment of clinical information systems in the healthcare setting: an efficient method for time-pressed evaluation. Methods Inf Med
16. Trotter RT, Needle RH, Goosby E, et al. A methodological model for rapid assessment, response, and evaluation: the RARE program in public health. Field Methods
17. Decision Logic Flow Charts and Knowledge Resource Reports. Available at: www.acoem.org/
18. Crabtree BF, Miller WL, eds. Doing Qualitative Research, 2nd ed. Thousand Oaks, CA: Sage; 1999.
19. Berg BL, Lune H. Qualitative Research Methods for the Social Sciences. 8th ed.Boston, MA: Pearson; 2012.
20. Ash JA, Chase D, Baron S, et al. Clinical decision support for worker health: a qualitative study of five primary care settings (podium presentation abstract). AMIA Annu Symp Proc
21. Garcia D, Hopewell J, Liebman A, Mountain K. The Migrant Clinicians Network: connecting practice to need and patients to care. J Agromed
22. Baron SL, Beard S, Davis LK, et al. Promoting integrated approaches to reducing health inequities among low-income workers: applying a social ecological framework. Am J Ind Med
23. Azaroff LS, Levenstein C, Wegman DH. Occupational injury and illness surveillance: conceptual filters explain underreporting. Am J Public Health
24. Socias C, Liang Y, Delclos G, Graves J, Hendrikson E, Cooper S. The feasibility of using electronic health records (EHRs) to describe demographic and clinical indicators of migrant and seasonal farmworkers. J Agromed
25. Ruser JW. Examining evidence on whether BLS undercounts workplace injuries and illnesses. Mon Labor Rev
26. Leigh JP, Marcin JP, Miller TR. An estimate of the U.S. Government's undercount of nonfatal occupational injuries. J Occup Environ Med
27. Rosenman KD, Kalush A, Reilly MJ, Gardiner JC, Reeves M, Luo Z. How much work-related injury and illness is missed by the current national surveillance system? J Occup Environ Med