Secondary Logo

Journal Logo

Analysis of Resident Case Logs in an Anesthesiology Residency Program

Yamamoto, Satoshi MD; Tanaka, Pedro MD, PhD; Madsen, Matias Vested MD; Macario, Alex MD, MBA

doi: 10.1213/XAA.0000000000000248
Case Reports: Education
Free

Our goal in this study was to examine Accreditation Council for Graduate Medical Education case logs for Stanford anesthesia residents graduating in 2013 (25 residents) and 2014 (26 residents). The resident with the fewest recorded patients in 2013 had 43% the number of patients compared with the resident with the most patients, and in 2014, this equaled 48%. There were residents who had 75% more than the class average number of cases for several of the 12 case types and 3 procedure types required by the Accreditation Council for Graduate Medical Education. Also, there were residents with fewer than half as many for some of the required cases or procedure types. Some of the variability may have been because of the hazards of self-reporting.

From the Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, Stanford, California.

Matias Vested Madsen, MD, is currently affiliated with the Department of Anaesthesiology, Herlev Hospital, University of Copenhagen, Copenhagen, Denmark.

Accepted for publication July 28, 2015.

Funding: None.

The authors declare no conflicts of interest.

Address correspondence to Satoshi Yamamoto, MD, Department of Anesthesiology, Perioperative and Pain Medicine, Stanford University School of Medicine, 300 Pasteur Dr., Room H3580, Stanford, CA 94305. Address e-mail to stcyamamoto@gmail.com.

Measuring the number of cases as a surrogate for clinical experience has been performed for decades.1 Caseload tracking also occurs outside the United States, including in Canada, the United Kingdom, Australia, and other countries.2–6 One reason to monitor cases performed by the trainee is that an increasing number of cases is associated with progression up the learning curve for some procedures.7 However, the optimal number of cases of a particular type that an anesthesia resident needs to perform to achieve proficiency is unknown. As a result, case volume alone is not sufficient to ensure adequate training.8

The electronic Accreditation Council for Graduate Medical Education (ACGME) Accreditation Data System resident case log system was designed and implemented to quantify the experiences of residents during their training.9 Anesthesiology residents are required to self-report all their cases and procedures by entering them manually into an online database. This is done in part to document that residents exceed the ACGME required minimum number in 12 different patient categories and 3 procedure types (spinal, epidural, and nerve block) to graduate anesthesia residency.

ACGME case log data have been used as decision support to help determine resident operating room (OR) assignments.10 Comparing ACGME case logs and information derived from the anesthesia information management system, it was found that residents often overcount or undercount their clinical caseload.11

Analyzing case numbers for anesthesia trainees within the same residency may yield insights about the variety in training experience within a program and the opportunities to improve the case log tracking system. In addition, more hospitals are asking programs to provide case numbers for specific case types (e.g., anesthesia for pediatric patients or for obstetric patients) and procedures (e.g., management of cardiopulmonary bypass) as a part of the hospital privileges application. The goal of this descriptive, retrospective study was to examine the resident caseload profiles in 2 separate cohorts of graduates of 1 residency program.

Back to Top | Article Outline

METHODS

This study was deemed exempt by our IRB as research conducted in an educational setting. Case logs as entered into the ACGME website for Stanford anesthesia residency graduates for 2013 (n = 25 residents) and 2014 (n = 26) were studied. Data entry into the ACGME website on case experience by these residents was done real time during their clinical training. The ACGME Accreditation Data System website was queried on February 14, 2015, to obtain the raw data on case experience for each resident. The types of rotations that each resident had during the 3-year residency were also counted. Multiple regression analysis and Pearson correlation coefficient were performed to assess predictors of case numbers.

Back to Top | Article Outline

RESULTS

The 2013 graduate cohort logged an average of 1373 patients (SD, 164; median, 1366; national average, 1280) during the residency and the 2014 cohort averaged 1323 patients (SD, 260; median, 1394; national average, 1234). The resident logging the fewest total patients (n = 808) in 2013 had 43.0% the number of patients compared with the resident with the most patients entered (n = 1877), and in 2014, this equaled 48.1% (806/1676; Figs. 1 and 2)

Figure 1

Figure 1

Figure 2

Figure 2

For 2013, on average, 83% (SD, 11%; median, 80%; range, 59%–100%) of all entered cases were required cases, and for 2014, this equaled 82% (SD, 11%; median, 82%; range, = 59%–100%). Overall, residents who recorded having more pediatric cases (P = 0.002), more nerve blocks (P = 0.002), more vaginal deliveries (P = 0.0032), and more pain evaluations (P = 0.0018) were all each associated with higher total case counts, whereas the number of cardiac cases, thoracic cases, intracerebral cases, intracerebral open cases, cesarean deliveries, epidurals, spinal cases, and major vascular cases were not associated with higher total case counts. A greater number of research rotations were also associated with fewer total cases (Fig. 3).

Figure 3

Figure 3

On average for each of the required case and procedure types, 72% of the 2013 residents and 67% of the 2014 residents had caseloads that were between 75% of the class average and 150% of the class average. For both 2013 and 2014, there were ≥1 residents who had 75% more than the class average of number of some of the 12 required case types or the 3 required procedure types (Tables 1 and 2). Also, in both graduation cohorts, there were residents who had fewer than half as many of the required case types or required procedures as the class mean.

Table 1

Table 1

Table 2

Table 2

For 2013, for the required case or procedure types, residents performed from 1.6 times (pediatrics aged <12 years) to 4.5 times (epidural) the ACGME minimum number. For 2014, depending on the requirement, residents performed from 1.5 times (pediatrics aged <12 years) to 4.2 times (epidural) the ACGME minimum number.

There were 7 residents in 2013 and 2 residents in 2014 who did <50% of the class average for pediatric < 3 months, whereas there were 2 residents in 2013 and 1 resident in 2014 who did >200% the class average.

For cardiac cases, 7 residents in 2013 (none in 2014) had <50% of the class average and 2 had >200% the class average (none in 2014). For nerve blocks, 2 residents in 2013 and 1 in 2014 reported <50% the class average, whereas 1 in 2013 had >200% the class average.

Variability in number of cardiac cases was correlated with the number of cardiac rotations the residents did (Fig. 4). A greater number of pediatric (r = 0.542, P < 0.0001) and obstetric (r = 0.556, P < 0.0001) rotations was also correlated with more of those case types. There was variability in peripheral nerve blocks despite every resident doing a single 1-month rotation.

Figure 4

Figure 4

There was 1 resident in 2013 and 4 residents in 2014 who did <50% of the class average for new pain evaluations, whereas there were no residents in 2013 and 2 residents in 2014 who did >200% the class average for new pain evaluations. In 2013, on average, 78% (SD, 11%; median, 77%; range = 55%–100%) of total patients logged in were OR cases, and 89% (SD, 3%; median, 89%; range = 82%–95%) of all OR cases were general anesthetics. In 2014, on average, 76% (SD, 8%; median, 78%; range = 55%–92%) of all patients logged were OR cases, and 89% (SD, 3%; median, 90%; range = 82%–94%) of OR cases were general anesthetics.

Back to Top | Article Outline

DISCUSSION

Although most anesthesia residents in this training program recorded similar numbers of cases, there are a few residents in each of the 2 graduating classes who had many more cases than their peers, and there are a few residents who had many less cases than their peers. In addition to variability in the total case experience, there was also variability in the number of cases and cases required for the required categories. This variability could have been because of several factors including (1) once a resident reaches the minimum requirements for case and procedures, they may be less motivated to log more of them, (2) lack of accepted guidelines for when to log cases (e.g., some residents log cases when a resident relieves another resident, but not all residents do this), (3) number of research months varies by trainee and reduces caseload because the trainee is out of the clinical environment, or (4) some residents may be more meticulous in recording of every case and procedure.

Approximately 70% of residents had caseloads between 75% of the class average and 150% of the class average. A few residents in each of the 2 graduation classes had many more (which we arbitrarily defined as 1.75 times the class average) pediatric cases or many more cardiac cases than their peers. For cardiac anesthesia, some residents completed as many as 6-month long rotations, whereas many residents completed the required minimum of 2 months. It may be that a few residents are able to customize their clinical training experience by choosing to do more elective months on rotations of particular interest resulting in many more of a particular clinical specialty case type compared with the class average. One or 2 residents in each graduating class did more than twice the class average of pediatric anesthesia.

The variability in case load that we observed is important, because it is a reminder that training programs cannot be evaluated simply by the average number of cases performed by the trainee. That summary data may not indicate whether a couple of residents are at the 90th percentile in cases number experience and another may be at the 10th percentile of experience.12

Although focusing on a particular clinical subspecialty during residency is not a formal pathway within our training program, the 2010 Carnegie study on medical education recommends creating individualized learning pathways.13 Depending on the case and volume mix at different institutions and training programs, house staff may be able to focus on some subspecialties if there is flexibility in the elective rotation schedule. As another example of residents customizing their training, a few residents in each cohort had 5 to 9 research months in the 3-year residency.

There are other published studies of case log analyses, mostly in the surgical disciplines. These studies have found that operative volume has decreased, increased, or stayed the same depending on the type of surgery.14 For example, a study of spine surgery procedure volumes determined that in orthopedic and neurosurgery residency training programs, caseload varies both within and among specialties.15 A few studies examined general surgery and showed that residents had fewer total cases,16 fewer pediatric surgery cases,17 and fewer open abdominal procedures18 after implementation of duty hour restrictions. Surgery resident comfort level with procedures was associated with performing more procedures.19 None of these studies specifically evaluated differences in case experience by residents within the same training program. The American Board of Anesthesiology does not consider case log reports for certification, whereas some surgical specialties do use them for their certification process.a,b

Future studies need to examine what fraction of case variability is because of differences in self-report and how much is related to residents choosing different clinical electives. Even though some residents in our study reported >1800 cases in 36 months of residency, it is possible that there were other additional cases and procedures that were not entered. For example, it may be that residents are biased toward entering those that are required and not entering more routine cases. As a result, the actual true case experience may be underreported because of this self-report bias. Other studies have demonstrated the potential for inaccuracies and discrepancies.20–22 Discrepancy between self-report and actual cases by surgery residents equaled 20%. Another study found that more than half of neurology residents entered less than half of their cases, and many residents indicated the case log system interfered with their education and patient care.21 Emergency medicine studies estimated that manual self-reporting results in capturing only 60% of actual clinical experience.c23

Given our findings and other investigators’ findings, it is likely worthwhile for residency leadership to establish guidelines on entering cases so that there is more uniformity in understanding and practice. These guidelines would need to specify among other things, for instance, that the thoracic case example should be entered once with the 3 required elements (thoracic, epidural, and nerve block) checked off on the website questionnaire and not entered 3 times separately with each required element (as we have heard is possible anecdotally). If various counted elements of a thoracic case were entered as separate patients, this would skew the total number of patients, but not change the count of the tracked elements, which form the primary basis of the case-tracking evaluation.

Education and instructions regarding case logs should include guidelines about logging cases with handoffs. The ACGME Anesthesiology FAQd does provide some guidance. For example, for the question, “Can two residents/fellows individually count the same case if they both participate in the patient’s care?” the document states: “If two individuals were involved in the majority of a major case (such as a liver transplant), including the most significant portions, each resident, or a resident and a fellow, can receive credit for the case by entering it into the Case Log System. When one resident or fellow completes a case for another, only the individual involved in the most significant aspects of the case, or the majority of the procedure, should record credit for the case.” Although these FAQs exist, it is not clear how widely disseminated they are to residents.

Rubrics on how to categorize particular cases such as new pain evaluations are also needed. For example, some residents may log a consult for analgesia in the postanesthesia care unit as a new pain evaluation, whereas others may only log a pain consult when on the acute pain rotation. In our study, some residents recorded less than half the class average for new pain evaluations, and some residents had >200% the class average. Another example of a possible area of documentation confusion for the resident is if a patient receives a nerve block by resident Smith in the preoperative area and resident Jones takes care of the patient in the OR. Sometimes resident Jones may report that the case had a nerve block instead of the resident performing the block, so there would be double counting.

Inaccuracies are also likely to occur when the time gap is longer between performing the case and entering the data. Some residents may feel pressure to report high case log numbers because of the minimum case number requirements and scrutiny by residency programs, the Resident Review Committee, and prospective employers. Because of the flaws with self-reporting, several programs have used innovative ways to improve the accuracy of reporting.22 It may be that applications for the smart phone may make it easier for house staff to enter cases. We are starting a project to extract cases and procedures directly from the electronic medical record.

There are no standards beyond the minimum case requirements as to how many other cases and procedures an anesthesia resident should do. Also, currently, the ACGME database does not collect the types of patients and conditions residents care for in the intensive care unit, preoperative clinics, or postanesthesia care unit, all of which are required rotations. Residents may enter procedures such as invasive lines or nerve blocks performed on those patients in the ACGME online database as another source of variability in procedure numbers.

Other items may be beneficial to collect experience on to better quantify a resident’s training. In fact, hospital privilege applications for new graduates may ask for confirmation of cases performed not currently in the ACGME data collection system. Actual examples of such requests received by our program include the resident case total for management of cardiac arrest, diagnostic tap, spinal drain, blood patches, acupuncture, fluoroscopy, spinal steroid injection, and transversus abdominis plane block.

Residents who recorded having more pediatric cases, nerve blocks, vaginal deliveries, and more pain evaluations had higher total case counts. This is likely partly explained because there is a potential for more cases in these areas in our training program. In contrast, the upper number of possible cardiac, thoracic, neurosurgery, and major vascular cases is more limited. Hence, those case types were not associated with higher total case counts.

Although the ACGME has minimums for 15 common anesthesia procedures required for resident graduation, there are no maximums stipulated above which a resident need not perform anymore. The relationship between case volume and achievement of milestones deserves further study. Although the overall case totals for the residents studied were similar to national averages as reported on the ACGME website, 1 limitation of this study is that case logging practice at 1 residency may not be representative of programs around the country.

In conclusion, analysis of resident case logs in our large anesthesiology residency program reveals variability in total required cases and procedures. Lacking objective data about cases performed versus cases reported by the house staff, we can only speculate about the causes of the variability. Some variability may be because of hazards of self-reporting and resulting bias, whereas some of the variability may be because of house staff choosing to complete different electives, thereby customizing their training even if there are no formal pathways. More research is needed to better understand which anesthesia cases and procedures are not being logged in each residency and nationally. More formalized and comprehensive guidelines to standardize case logging practices may be especially useful because employers ask for these data and because ACGME increasingly evaluates residencies based on the data collected electronically.

Back to Top | Article Outline

FOOTNOTES

a ABPlSurg.org. Available at: https://www.abplsurg.org/ModDefault.aspx?ReturnUrl=%2f. Accessed June 23, 2015.
Cited Here...

b The American Board of Anesthesiology. Available at: http://www.theaba.org/Home. Accessed June 23, 2015.
Cited Here...

c Lee JS, Sineff SS, Sumner W. Validation of electronic student encounter logs in an emergency medicine clerkship. In: Proceedings of the AMIA Symposium. American Medical Informatics Association, 2002:425. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2244433/. Accessed April 16, 2015.
Cited Here...

d https://www.acgme.org/acgmeweb/Portals/0/PDFs/FAQ/040_anesthesiology_FAQs_07012014.pdf. Accessed June 23, 2015.
Cited Here...

Back to Top | Article Outline

REFERENCES

1. Grasberger RC, McMillian TN, Yeston NS, Williams LF, Hirsch EF. Residents’ experience in the surgery of trauma. J Trauma. 1986;26:848–50
2. Stanley MD, Davenport DL, Procter LD, Perry JE, Kearney PA, Bernard AC. An acute care surgery rotation contributes significant general surgical operative volume to residency training compared with other rotations. J Trauma. 2011;70:590–4
3. Compeau C, Tyrwhitt J, Shargall Y, Rotstein L. A retrospective review of general surgery training outcomes at the University of Toronto. Can J Surg. 2009;52:E131–6
4. Kohn GP, Nikfarjam M. The effect of surgical volume and the provision of residency and fellowship training on complications of major hepatic resection. J Gastrointest Surg. 2010;14:1981–9
5. Drake FT, Van Eaton EG, Huntington CR, Jurkovich GJ, Aarabi S, Gow KW. ACGME case logs: surgery resident experience in operative trauma for two decades. J Trauma Acute Care Surg. 2012;73:1500–6
6. Cronenwett JL, Liapis CD. Vascular surgery training and certification: an international perspective. J Vasc Surg. 2007;46:621–9
7. Konrad C, Schüpfer G, Wietlisbach M, Gerber H. Learning manual skills in anesthesiology: is there a recommended number of cases for anesthetic procedures? Anesth Analg. 1998;86:635–9
8. Starke RM, Asthagiri AR, Jane JA Sr, Jane JA Jr. Neurological surgery training abroad as a progression to the final year of training and transition to independent practice. J Grad Med Educ. 2014;6:715–20
9. Salazar D, Schiff A, Mitchell E, Hopkinson W. Variability in Accreditation Council for Graduate Medical Education Resident Case Log System practices among orthopaedic surgery residents. J Bone Joint Surg Am. 2014;96:e22
10. Wanderer JP, Charnin J, Driscoll WD, Bailin MT, Baker K. Decision support using anesthesia information management system records and accreditation council for graduate medical education case logs for resident operating room assignments. Anesth Analg. 2013;117:494–9
11. Simpao A, Heitz JW, McNulty SE, Chekemian B, Brenn BR, Epstein RH. The design and implementation of an automated system for logging clinical experiences using an anesthesia information management system. Anesth Analg. 2011;112:422–9
12. Veldenz HC, Dennis JW, Dovgan PS. Quality control of resident operative experience: compliance with RRC criteria. J Surg Res. 2001;98:81–4
13. Irby DM, Cooke M, O’Brien BC. Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85:220–7
14. Simien C, Holt KD, Richter TH, Whalen TV, Coburn M, Havlik RJ, Miller RS. Resident operative experience in general surgery, plastic surgery, and urology 5 years after implementation of the ACGME duty hour policy. Ann Surg. 2010;252:383–9
15. Daniels AH, Ames CP, Smith JS, Hart RA. Variability in spine surgery procedures performed during orthopaedic and neurological surgery residency training: an analysis of ACGME case log data. J Bone Joint Surg Am. 2014;96:e196
16. Kairys JC, McGuire K, Crawford AG, Yeo CJ. Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees? J Am Coll Surg. 2008;206:804–11
17. Gow KW, Drake FT, Aarabi S, Waldhausen JH. The ACGME case log: general surgery resident experience in pediatric surgery. J Pediatr Surg. 2013;48:1643–9
18. McCoy AC, Gasevic E, Szlabick RE, Sahmoun AE, Sticca RP. Are open abdominal procedures a thing of the past? An analysis of graduating general surgery residents’ case logs from 2000 to 2011. J Surg Educ. 2013;70:683–9
19. Suwanabol PA, McDonald R, Foley E, Weber SM. Is surgical resident comfort level associated with experience? J Surg Res. 2009;156:240–4
20. Rosenberg TL, Franzese CB. Extremes in otolaryngology resident surgical case numbers. Otolaryngol Head Neck Surg. 2012;147:261–70
21. Gill DJ, Freeman WD, Thoresen P, Corboy JR. Residency training the neurology resident case log: a national survey of neurology residents. Neurology. 2007;68:E32–3
22. Bhattacharya P, Van Stavern R, Madhavan R. Automated data mining: an innovative and efficient web-based approach to maintaining resident case logs. J Grad Med Educ. 2010;2:566–70
23. Langdorf MI, Montague BJ, Bearie B, Sobel CS. Quantification of procedures and resuscitations in an emergency medicine residency. J Emerg Med. 1998;16:121–7
© 2016 International Anesthesia Research Society