A previously described innovative, multidisciplinary model involving collaboration with a neurophysiologist to provide anesthesiology residents with additional electroencephalogram (EEG) experience with and through interpretation of EEGs was designed1 2
METHODS
The University of Kentucky IRB and the Graduate Medical Education Committee approved this prospective study. This study was conducted with written informed consent from the study subjects. During 1-month neurosurgical intensive care unit rotation from December 2007 through June 2010, anesthesiology residents who agreed to participate were included in the study with sequential enrollment. No more than 2 residents were on the neurosurgical intensive care unit rotation in a month. Twenty residents participated; each group had 3 participants in postgraduate year 3 (PGY3), and 7 participants in postgraduate year 2 (PGY2).
The same curriculum as previously described1 , 2 2 Table 1 ). Long-term retention was targeted for 12 months after the curriculum. Long-term retention was assessed using an evaluation tool comprising 40 of the best-performing items from the previous assessment tools (Table 1 ) with most questions from the baseline assessment tool. All the 25-item assessments were unique to that specific time period for each administration, with all participants completing the same assessment tool at the same time points during the study. A 40-item assessment tool consisted of the same items for all participants, and questions were chosen from the examinations previously administered. Sequential enrollment involved the 20 EEG group first, followed by the 10 EEG group.
Table 1: Content of Electroencephalogram (EEG) Evaluation Tools
An independent third party scored all the evaluation tools, thereby blinding the investigators to the results. Evaluation scores are the percentage of correct answers.
Statistical Analyses
For a sample size n = 10, a minimal detectable difference between mean EEG examination percentage correct scores was calculated to be 9 percentage points at a power of 80% (α = 0.05), assuming the standard deviation (SD) to be 8. Mixed model analysis, which allows for the modeling of repeated measurements of scores obtained over time from each subject, was performed to examine the effect of the group and time on evaluation scores and how the group means differed at the different time points by including time category (with 3 levels: baseline, after 10 EEG interpretations, and long term), group (with 2 levels: 10 and 20 EEG), and their interaction in the model (Model A). Alternatively, we fit a similar model including scores from the baseline test as a covariate in the model to assess the group effect after 10 EEG interpretations and at long-term adjusting for the effect of baseline knowledge (Model B). Bonferroni adjustments were used for multiple comparisons. All significance tests were 2-sided, setting α to be 0.05. All statistical analyses were performed with SAS (version 9.3, Cary, NC).
RESULTS
There were 20 anesthesiology residents who consented and completed the required number of EEG interpretations and long-term retention assessment. Each group consisted of 10 residents. The significance of predictors included in each model is shown in Table 2 , with significant interaction between time and group, suggesting differing group effects depending on time points. The mean scores for the 10 EEG group went from 42.8% ± 14.4% at baseline to 63.2% ± 8.0% (P < 0.0001) after 10 EEG interpretations. However, it reduced to 49.3% ± 9.9% for long-term retention using Model A. There was no evidence of a significant difference in scores between baseline and long-term (6.9% ± 9.0%, P = 0.78). In the 20 EEG group, the mean score improved from 34.4% ± 9.7% at baseline to 63.2% ± 6.2% (P < 0.0001) after 10 EEGs, but there was no evidence of a significant reduction −1.0% ± 5.9% (P = 1.00). Hence, the mean score for the 20 EEG group for long-term retention was significantly higher than the mean baseline score (P < 0.0001) (Table 3 ). Differences between the 2 groups were noted only at the point of long-term assessment (P = 0.006), with 20 EEG interpretations compared with 10 using Model A (Fig. 1 ). There was no evidence of a significant difference between the groups (Fig. 2 ) at baseline (P = 0.0652) or after 10 EEG interpretations (P = 1.00). When adjusted for scores from baseline test using Model B, similar results were obtained for differences between groups, observing only significant differences at long term (P = 0.0002). There was no significant reduction from score after 10 EEG interpretations to long-term score for the 20 EEG group (P = 0.7432), whereas reduction was significant for the 10 EEG group (P = 0.0001). We obtained similar results using either model as summarized in Table 3 .
Table 2: Significance of Variables in the Model for Model A and Model B
Table 3: Descriptive Statistics and P Values for Comparisons of Scores Between Groups and Time Points
Figure 1: Mean electroencephalogram (EEG) evaluation scores (percentage correct) for each time point: before EEG reading (baseline), after 10 EEG interpretations, and long term, stratified by group status. *Indicates significant differences (P < 0.01) for a given comparison, with each comparison represented by a double-headed arrow (↔).
Figure 2: Electroencephalogram (EEG) evaluation results (10 vs 20 EEG group).
Figure 3 depicts how the relationships between EEG scores differed between the groups and across time points. Baseline scores were weakly associated with scores after 10 EEGs for the 10 EEG group (r = 0.22, P = 0.267), and there was a trend for an association in the 20 EEG group (r = 0.51, P = 0.065) in the expected positive direction (Fig. 3A ). However, the weak correlation in the 10 EEG group may have been unduly influenced by 1 outlying pairwise association. The correlation between baseline scores and scores after 10 EEGs in the 10 EEG group after removing that outlier were stronger (r = 0.65, P = 0.027). Baseline scores were positively correlated with scores at long term for the 10 EEG group (r = 0.78, P = 0.004), with a trend for a positive association in the 20 EEG group (r = 0.53, P = 0.059; Fig. 3B ). The largest group differences occurred for the association between scores after 10 EEGs and scores at long term (Fig. 3C ). The relationship between these scores was stronger in the 20 EEG group (r = 0.79, P = 0.004) compared with the 10 EEG group (r = 0.20, P = 0.286). Residual plot and influential plots did not show any serious departures from the assumptions of the mixed models. There was no evidence of violation of normality assumption according to the Kolmogorov-Smirnov test (P > 0.15).
Figure 3: Scatterplots of relationships among electroencephalogram (EEG) examination scores across different time points, stratified by group status (10 vs 20 EEG: A) Scores at baseline vs scores after 10 EEGs; B) Scores at baseline vs scores at long term (12 months); and C) Scores after 10 EEGs vs scores at long term. Lines represent trend lines for each relationship, stratified by group status.
DISCUSSION
The published literature involving the educational methods of EEG instruction is sparse with some recommendations for the EEG curriculum1–3 4 , 5 1 , 2 , 6 , 7 2
Clinical decision making and problem solving require a core of basic factual knowledge. A primary goal of teaching material is to enable a student to remember the material for an extended period of time and have it accessible when it is needed to make clinical decisions. Studies showing initial learning of the material by residents have shown significant memory lapse over a 6- to 7-month period.8 9–12
In this EEG educational model, teaching sessions lasted approximately 1 hour with an average of 5 EEG interpretations per session; this entailed 2 and 4 sessions for the 10 EEG group and 20 EEG groups, respectively. These sessions were scheduled weekly (20 EEG group) or twice monthly (10 EEG group) during the rotation, and occasionally, because of the availability of the residents and/or clinical neurophysiologist, these sessions occurred twice per week. One of the advantages of this EEG educational model may have involved the spaced distribution of educational encounters known in cognitive psychology as the “spacing effect.” The spacing effect refers to the idea that information is more effectively learned and retained when educational opportunities are provided with a spaced distribution presented at different times and repeated at specific intervals rather than presentation as a single bolus of educational material.13–15 16 17
Cognitive psychology suggests that additional processing and specifically repeated retrieval of the information after initial learning may play a role in long-term retention.18 19 19 20–24 25
To remove a confounding variable, feedback was not provided after any assessment. Providing assessment feedback improves retention of the material as it corrects errors and confirms correct responses.26–29
This study has several limitations. The modest sample size is one of these limitations although statistical significance was observed despite the limited sample size. We could not perform a factor analysis in this study, but we plan to in future studies. It is possible that the 20 EEG group may have had additional EEG exposure during the period between completion of the EEG teaching module and the long-term retention assessment, giving them an advantage and possibly improving their scores. This is unlikely, as minimal EEG exposure occurred intraoperatively in the interim, and any exposure should have affected both groups equally.
It is also possible that the 20 EEG group had an advantage because of their longer-term exposure to knowledge that may take longer to learn. This could be demonstrated by the evaluation tools, thus improving their long-term scores. However, we find this unlikely for 2 reasons. The first is that, in a prior study, the 20 EEG group did not show significant knowledge gain after 10 EEG readings, which leads us to believe that the extra EEG exposures did not bolster this type of knowledge and that the evaluation tools did not assess for this type of knowledge although present, or both. The second is that the long-term evaluation tool is a composite of questions from the prior evaluation tools with most items drawn from the baseline tool. It has the same structure and measures the knowledge from the same core content targeted in the previous evaluation tools. Because of this absence of change in the evaluation tool design and content, it is unlikely that the design of the evaluation tools would explain the results.
Another explanation for the difference could have involved assessment performance differences between the 2 groups. Learning styles were not assessed and could have impacted examination performance.29 , 30
This study assessed knowledge applied at a higher level, the second level of Miller’s pyramid hierarchy of evaluation.31
DISCLOSURES
Name : Brenda G. Fahy, MD.
Contribution : This author helped design and conduct the study, analyze the data, and write the manuscript.
Attestation : Brenda G. Fahy has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.
Conflicts of Interest : Brenda G. Fahy received research funding from Aspect Medical. She is involved as a mentor and coinvestigator in an industry sponsored data collection project with Aspect Medical for acute kidney injury.
Name : Destiny F. Chau, MD.
Contribution : This author helped conduct the study and write the manuscript.
Attestation : Destiny F. Chau has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Conflicts of Interest : Destiny F. Chau reported no conflicts of interest.
Name : Tezcan Ozrazgat-Baslanti, PhD.
Contribution : This author helped analyze the data and write the manuscript.
Attestation : Tezcan Ozrazgat-Baslanti has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Conflicts of Interest : The author has no conflicts of interest to declare.
Name : Meriem Bensalem Owen, MD.
Contribution : This author helped conduct the study, analyze the data, and write the manuscript.
Attestation : Meriem Bensalem Owen has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Conflicts of Interest : Meriem Bensalem Owen received research funding from UCB, Lunbeck, and Sunovion. She is involved as an investigator in industry sponsored drug trials with UCB, Lundbeck, and Sunovion.
This manuscript was handled by : Franklin Dexter, MD, PhD.
ACKNOWLEDGMENTS
Special thanks to Osborne Hall and Corey Astrom for their assistance in preparation of this manuscript.
REFERENCES
1. Fahy BG, Chau DF, Owen MB. The effectiveness of a simple novel approach on electroencephalograph instruction for anesthesiology residents. Anesth Analg. 2008;106:210–4, table of contents
2. Fahy BG, Chau DF, Bensalem-Owen M. Evaluating the requirements of electroencephalograph instruction for anesthesiology residents. Anesth Analg. 2009;109:535–8
3. . Syllabus for clinical electroencephalographers. J Clin Neurophysiol. 1994;11:133–42
4. Mayorov OY. Virtual training simulator–designer of EEG signals for tutoring students and doctors to methods of quantitative EEG analysis (qEEG). Stud Health Technol Inform. 2000;77:573–7
5. de Beer NA, van Meurs WL, Grit MB, Good ML, Gravenstein D. Educational simulation of the electroencephalogram (EEG). Technol Health Care. 2001;9:237–56
6. Chau DF, Bensalem-Owen MK, Fahy BG. The effectiveness of an interdisciplinary approach to EEG instruction for residents®. J Clin Neurophysiol. 2010;27:106–9
7. Bensalem-Owen M, Chau DF, Sardam SC, Fahy BG. Education research: evaluating the use of podcasting for residents during EEG instruction: a pilot study. Neurology. 2011;77:e42–4
8. Larsen DP, Butler AC, Roediger HL 3rd. Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial. Med Educ. 2009;43:1174–81
9. Ebbinghaus HRuger HA, Bussenius CE. über das Gedächtniss. Leipzig 1885. Memory: A Contribution to Experimental Psychology. trans-ed. 1913; section 29 & 34. Available at:
http://psychclassics.yorku.ca/Ebbinghaus/index.htm . Accessed 26 November 2012
10. Hovland CIStevens SS. Human learning and retention. Handbook of Experimental Psychology. 1951 New York, NY John Wiley & Sons:645–6 In:
11. McGeoch JA, Irion AL The Psychology of Human Learning, 2nd Ed. 1952 New York, NY Van Rees Press:355–6
12. Searleman A, Herrmann D Memory from a Broader Perspective. 1994 New York, NY McGraw-Hill:99–101
13. Bjork RAGruneberg MM, Morris PE, Sykes RN. Retrieval practice and the maintenance of knowledge. Practical Aspects of Memory: Current Research and Issues. 1988 New York, NY John Wiley & Sons:396–401
14. Glenberg AM, Lehmann TS. Spacing repetitions over 1 week. Mem Cognit. 1980;8:528–38
15. Pashler H, Rohrer D, Cepeda NJ, Carpenter SK. Enhancing learning and retarding forgetting: choices and consequences. Psychon Bull Rev. 2007;14:187–93
16. Moulton CA, Dubrowski A, Macrae H, Graham B, Grober E, Reznick R. Teaching surgical skills: what kind of practice makes perfect?: a randomized, controlled trial. Ann Surg. 2006;244:400–9
17. Kerfoot BP, Baker HE, Koch MO, Connelly D, Joseph DB, Ritchey ML. Randomized, controlled trial of spaced education to urology residents in the United States and Canada. J Urol. 2007;177:1481–7
18. Leeming FC. The exam-a-day procedure improves performance in psychology classes. J Edu Psychol. 1951;42:1–20
19. Larsen DP, Butler AC, Roediger HL 3rd. Test-enhanced learning in medical education. Med Educ. 2008;42:959–66
20. Roediger HL, Karpicke JD. Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci. 2006;17:249–55
21. McDaniel MA, Anderson JL, Derbish MH, Morrisette N. Testing the testing effect in the classroom. Eur J Cogn Psychol. 2007;19:494–513
22. Butler AC, Roediger HL 3rd. Testing improves long-term retention in a simulated classroom setting. Eur J Cogn Psychol. 2007;19:514–27
23. Tulving E. The effects of presentation and recall of material in free recall learning. J Verbal Learn and Verbal Behav. 1967;6:175–84
24. Karpicke JD, Roediger HL 3rd. The critical importance of retrieval for learning. Science. 2008;319:966–8
25. Kromann CB, Jensen ML, Ringsted C. The effect of testing on skills learning. Med Educ. 2009;43:21–7
26. McDaniel MA, Fisher RP. Tests and test feedback as learning sources. Contemp Educat Psychol. 1991;16:192–201
27. Bangert-Drowns RL, Kulik JA, Kulik CC. Effects of frequent classroom testing. J Educat Res. 1991;85:89–99
28. Pashler H, Cepeda NJ, Wixted JT, Rohrer D. When does feedback facilitate learning of words? J Exp Psychol Learn Mem Cogn. 2005;31:3–8
29. McNulty JA, Sonntag B, Sinacore JM. Evaluation of computer-aided instruction in a gross anatomy course: a six-year study. Anat Sci Educ. 2009;2:2–8
30. McManus IC, Richards P, Winder BC, Sproston KA. Clinical experience, performance in final examinations, and learning style in medical students: prospective study. BMJ. 1998;316:345–50
31. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65:S63–7
