Skip Navigation LinksHome > October 2006 - Volume 81 - Issue 10 > Do Students Do What They Write and Write What They Do? The M...
Academic Medicine:
Measuring Clinical Skills

Do Students Do What They Write and Write What They Do? The Match between the Patient Encounter and Patient Note

Szauter, Karen M.; Ainsworth, Michael A.; Holden, Mark D.; Mercado, Anita C.

Section Editor(s): Mechaber, Alex MD; Albanese, Mark PhD

Free Access
Article Outline
Collapse Box

Author Information

Correspondence: Karen Szauter, MD, The University of Texas Medical Branch, 301 University Blvd, Galveston, Texas 77555-0420; e-mail: (kszauter@utmb.edu).

Collapse Box

Abstract

Background: Patient notes are used for a variety of purposes in health care. Medical students are taught the structure of patient notes early in training. Review of patient notes are then used to assess synthesis and integration of patient information. It is critical that the information in the note accurately and completely represents the student-patient encounter.

Method: The authors reviewed videotapes of students in three standardized-patient based scenarios and compared what occurred during the physical examination with the subsequent documentation in the patient note.

Results: In all, 207 encounter-note pairs were reviewed. Only 8 (4%) of the notes completely and accurately represented what occurred during the encounter. Problems with underdocumentation, overdocumentation, and inaccurate documentation of physical findings were seen for all three patient scenarios.

Conclusions: These findings highlight the need to teach and assess both data gathering skills and written documentation of findings in medical training.

Written documentation of the patient encounter serves many essential roles in health care. Patient notes chronicle important findings relating to health and disease, provide communication between health care providers, and are the documents used for billing. Patient notes are also used for epidemiological research, to study practice patterns of physicians, or as a primary source of information during malpractice litigation.1–3 Well-organized, complete, and accurate documentation of what occurs between a health care professional and a patient is essential.

Learning the skills for written documentation of patient encounters begins during medical school. The Medical School Objectives Project (MSOP) recognizes written communication as an important skill, and assessment of written communication is included within the six Accreditation Council for Graduate Medical Education competencies.4,5 Time is devoted in medical curricula to teaching and refining this skill.6 Many clinical clerkships require students to submit patient write-ups as part of the evaluation process. Write-ups are reviewed in great detail, and attention is given to completeness and synthesis of patient information.7 However, this review is typically performed at a time remote from the actual patient encounter, with the underlying assumption that the data presented are accurate and representative of the student-patient interaction.

A limited number of studies have compared information obtained during patient encounters and the corresponding notes. One measurement method employed has been to obtain information directly from patients. Reports, gathered through interviews or surveys, are then compared to documentation from the patient visit. These studies often focus on a specific aspect of the encounter, and patient questioning is directed. The reliability of this technique depends on the patient’s understanding of the parameters being measured and patient recall.8 Completeness or accuracy of documentation has also been studied by comparing checklists scored by standardized patients (SPs) with associated patient notes. Studies using SPs have been performed in controlled testing situations or through the use of unannounced SPs in a practice setting. Discrepancies between what the SPs report as “done” and patient-note documentation for encounters have been identified; however findings in these studies are similarly limited by the specificity of the checklist items and accuracy of the SPs.9,10

The present study was designed to allow a direct comparison between student interactions with a patient and the subsequent patient note. For purposes of this study, we focused on the physical examination (PE). Our goal was to provide a direct comparison between the PE maneuvers that were performed on the patient, and the corresponding documentation of the PE in the patient note. It is important to recognize that we focused on the complete and accurate recording of all PE maneuvers performed by students, rather than focusing only on examination maneuvers that were specific to the presenting problem portrayed by the patient.

Back to Top | Article Outline

Method

We used information from our senior medical student standardized patient-based clinical skills assessment (CSA) to address our question. This study was conducted though the review of videotaped student-SP encounters and the corresponding patient notes. Permission for this work was obtained from our Institutional Review Board and Curriculum Research Committee.

Our CSA has been in place for over 15 years, and achieving a passing score is part of the school’s graduation requirement. For the 2005 examination, each student participated in nine 15-minute standardized patient-based encounters. Following their second, fourth, sixth and eighth patient encounter, students were given 10 minutes to complete a patient note. The note template included sections for the medical interview, physical examination, differential diagnosis, and management plan, a format similar to that used in a nationally administered clinical skills assessment.11 Students were familiar with the examination structure and note format from a detailed orientation session held several weeks before the start of the CSA, and a brief orientation on the day of their exam.

The examination blueprint was aligned with objectives from the core clinical rotations. Patient scenarios were developed locally or adapted from existing SP case banks. Sixteen scenarios were prepared for the 2005 examination from which we selected three for this study. We chose adult cases with problems focused on gastroenterological (40-year-old man presenting with laboratory abnormalities in his liver profile), cardiovascular (44-year-old woman with intermittent tachyarrhythmias) and upper respiratory (55-year-old man with a persistent sore throat and chronic cough) symptoms. For each of these cases, students were instructed to perform a focused medical interview and physical examination. The SPs portraying these patients were experienced from previous work in our SP program and had no abnormal physical findings per the Medical Director’s physical assessment. All three cases had been used in our CSA in previous years.

Four investigators (the authors) participated in the study. All four are internal medicine faculty at our institution and are actively involved in teaching and assessing medical students. The investigators met prior to beginning the study to discuss guidelines for data collection. A well-accepted textbook of physical examination instruction, used for teaching at our institution, served as the standard for PE techniques.12

In the pilot phase of the study, ten complete student-SP encounters were reviewed and were compared to the medical interview and PE portions of the patient note. Because of wide variations in interview styles, encounter-note agreement for the medical interview was difficult to assess objectively. We therefore chose to focus this study on the PE portion of the encounter and the corresponding section of the patient note. We assessed the potential influence of time constraints on student performance by recording time use in each encounter. We recorded time use from the beginning to the end of the encounter, and from initiation to completion of the PE component.

The principle author (KS) reviewed each patient encounter-note pair along with one of the other investigators. Reviewers were provided with copies of the tapes and watched the PE portion of the encounter. There was no limitation on the number of times that an encounter could be viewed. Details of all PE maneuvers performed were transcribed, then compared to the student’s written description of the PE in the corresponding note. Five scoring categories, defined prospectively by the investigators, were used to categorize the relationship between the patient encounter and corresponding note: (1) all PE maneuvers correctly recorded; (2) PE maneuver performed, but not recorded; (3) PE maneuver not performed, but recorded; (4) PE maneuver performed incorrectly, recorded as if done; and (5) inaccurate documentation (inclusion of an abnormal physical finding that was not present). For each patient encounter-note pair, all applicable categories could be selected. Reviews were performed independently and scoring categories were compared. If a category was selected by only one of the reviewers, transcribed details of the encounter were compared between reviewers and consensus was reached through discussion.

We performed a descriptive analysis of our findings by case, and calculated the frequency with which each scoring category was selected for the encounter-note pairs. For individual patient encounters, we also calculated the total encounter time used by each student, and the portion of time spent in the PE. Individual time use information was then used to calculate the mean total encounter time for each case, and the mean percent of time spent in the PE.

Back to Top | Article Outline

Results

Videotapes for all of the encounters with corresponding notes were obtained and reviewed. Technical problems resulted in the exclusion of 7 encounters, with a total of 207 encounter-note pairs available for the study. The frequency of the scoring categories noted for each case is presented in Table 1.

Table 1
Table 1
Image Tools

Almost all of the encounter-note pairs reviewed (96%) revealed some degree of mismatch between what students did during the PE and what they recorded in the note. Omissions of information were common, and included a wide range of PE maneuvers. Most concerning was that the majority of notes included documentation of findings from maneuvers that were either not performed during the PE (e.g., “lungs clear to auscultation bilaterally” when the lungs had not been examined; “no splenomegaly” when the examination of the abdomen had been limited to nonspecific, generalized palpation), or performed incorrectly such that the information reported could not have been reliably obtained (e.g., “PMI nondisplaced” when palpation of the chest wall had been performed on top of breast tissue). Of the 207 encounter-note pairs reviewed, 169 (82%) were found to include information from examination maneuvers that had either not been performed or had been performed incorrectly.

Documentation of PE abnormalities that were not actually present most often included findings relevant to the case content. For example, students recorded the presence of scleral icterus, ascites, or a grossly abnormal liver size for the patient with the abnormal liver profile, peripheral edema in the patient with the cardiac presentation, and cervical lymphadenopathy in the patient with upper respiratory symptoms. None of the SPs had these actual abnormalities.

Time use was similar across the three cases, as was the percent of time spent in the PE. Mean total time (standard deviation) for each case was: gastroenterological 13:39 (0:50), cardiovascular 13:53 (0:57), and respiratory 13:32 (1:16). The mean percent of time spent in the physical examination was 20.3% (6.7), 22.2% (7.6) and 22.5% (6.6) respectively.

Back to Top | Article Outline

Discussion

Our findings revealed a high rate of mismatch between what students did during the PE portion of patient encounters and what they documented in the subsequent patient note. The potential underlying reasons and implications of these findings are complex. For purposes of discussion, we chose to globally classify the mismatch errors into three categories: underdocumentation, overdocumentation, and inaccurate documentation.

Several studies have evaluated issues relating to underdocumentation and why this may occur.8–10,13 Potential explanations for why our students did not record of all of the PE maneuvers that they performed include: they simply forgot that they had performed the maneuver, forgot to record it, or decided after the fact that the maneuver was not relevant and did not need to be recorded. While these are seemingly innocent errors of omission, failing to record information may subsequently underestimate the quality of care provided, can lead to medical errors, or can result in note with inadequate data to justify a selected level of billing.8,9,13,14

The problem of overdocumentation, widely seen in our study, is of greater concern. In the strictest sense, documenting information that was not obtained constitutes fraud, and has wide reaching implications for professionalism. Because this was a multistation examination, students may have become confused and documented findings with an honest belief that they had performed the maneuver on the patient they had just seen. Others, however, may have chosen to embellish the note, recognizing after the encounter that there were parts of the examination that should have been performed. This latter behavior is clearly unacceptable in any setting.15 As more health care systems transition to template-based history and PE forms or electronic medical records, the potential to overdocument the patient encounter increases.14,15 Issues of overdocumentation have not been well studied and represent an opportunity for emphasis in students’ professional development.

The other study category representing overdocumentation was that of students who performed the PE incorrectly, then recorded information that was not justified by what had been done. The frequency of this finding raises concern about students’ actual knowledge of, or the ability to perform, correct PE techniques. Our curriculum includes detailed teaching of the physical examination in the preclinical years, and all students must successfully demonstrate a comprehensive PE on an SP with direct faculty observation. However, deficiencies in students’ physical examination skills have been well described in the recent literature, with some suggestion that students’ examination skills decline during clinical training.16 We speculate that students may have thought that their technique was adequate to support what they recorded. An alternative explanation is that students intentionally used inferior technique in an attempt to save time. Our experience suggests that the case challenges were reasonable for the time allotted; however, direct feedback from students would be helpful in assessing the influence of time on their performance.

Problems with inaccurate documentation (recording PE abnormalities that were not present) were most often case specific, implying that students were led astray by the case context. This is a concerning observation with dangerous implications in patient care. Inaccurate recognition and documentation of abnormalities can lead to inappropriate testing or treatment choices, confusion by subsequent health care providers, and medical errors.

This study begins to investigate a problem that has received limited prior attention. Our restricted sample size and the setting from which the data were gathered may limit the generalizability of our findings. Ideally, one would like to abstract patient notes immediately after watching learners or physicians in an actual clinical setting, but practical and ethical issues limit such studies. Further work with unannounced standardized patients may be beneficial for investigations in this area.3,9,10

Our findings have several educational implications. Although the foundations for physical examination techniques and patient notes are taught during the preclinical years, these skills must be reinforced throughout clinical training. Ensuring that students can correctly identify and document physical findings is equally important. In addition, students need to be reminded of how written records are used in practice, and of their professional responsibility to document only what truly occurs during their encounters with patients.

Back to Top | Article Outline

Acknowledgments

The authors wish to thank Judy Thornton, Martha Phipps, and Hazel Smith for preparation of the standardized patients.

Back to Top | Article Outline

References

1 Solomon DH, Schaffer JL, Katz JN, et al. Can history and physical examination be used as markers of quality?: An analysis of the initial visit notes in musculoskeletal care. Med Care. 2000;38:383–91.

2 Politi BJ, Arena VC, Schwerha J, Sussman N. Occupational medical history taking: how are today’s physicians doing? A cross-sectional investigation of the frequency of occupational history taking by physicians in a major teaching hospital. J Occup Environ Med. 2004;46:550–55.

3 Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med. 2000;108:642–49.

4 The Medical School Objectives Writing Group. Learning Objectives for Medical Student Education—Guidelines for Medical Schools: Report I of the Medical School Objectives Project. Acad Med. 1999;74:13-8.

5 ACGME Outcome Project (http://www.acgme.org/outcome/comp/compFull.asp). Accessed 17 February 2006.

6 McCarty T, Parkes MV, Anderson TT, et al. Improved patient notes from medical students during web-based teaching using faculty- calibrated peer review and self-assessment. Acad Med. 2005;80:S67–S70.

7 Kogan JR, Shea J. Psychometric Characteristics of a write-up assessment form in a medicine core clerkship. Teach Learn Med. 2005;17:101–6.

8 Cohen RJ, Ek K, Pan CX. Complementary and alternative medicine (CAM) use by older adults: a comparison of self-report and physician chart documentation. J Gerontol Med Sci. 2002;57:M223–7.

9 Dresselhaus TR, Peabody JW, Lee M, Wang MM, Luck J. Measuring compliance with preventive care guidelines. Standardized patents, clinical vignettes, and the medical record. J Gen Intern Med 2000;15:782–8.

10 MacMillan MK, Fletcher EA, de Champlain AF, Klass D. Assessing post-encounter note documentation by examinees in a field test of a nationally administered standardized patient test. Acad Med. 2000;75:S112–4.

11 Practice Patient Notes (http://www.usmle.org/orientation/2006/PatientNote/PNEntry.htm). Accessed 11 February 2006.

12 Bickley LS, Szilagyi PG. Bate’s Guide to the Physical Examination and History Taking. 8th Ed. Philadelphia: Lippincott Williams & Wilkins, 2003.

13 Ellis PM, Blackshaw G, Purdie GL, Mellsop GW. Clinical information in psychiatric practice: what do doctors know, what do they think is known and what do they record? Med Educ. 1991;25:438–43.

14 Sprtel SJ, Zlabek JA. Does the use of standardized history and physical forms improve billable income and resident physician awareness of billing codes? South Med J. 2005;98:524–7.

15 Dresselhaus TR, Luck J, Peabody JW. The ethical problem of false positives: a prospective evaluation of physician reporting in the medical record. J Med Ethics. 2002;28:291–4.

16 York NL, Niehaus AH, Markwell SJ, Folse JR. Evaluation of students’ physical examination skills during their surgery clerkship. Am J Surg. 1999;177:240–3.

© 2006 Association of American Medical Colleges

Login

Article Tools

Images

Share