Feedback from online, anonymous questionnaires revealed that 224 (90%) of 248 respondents would recommend the programme to their peers and the same percentage reported that they had a better understanding of the basic and applied surgical sciences than they would have otherwise had if they had not enrolled on the ESSQ programme. The majority of students (81%) considered that the ESSQ would improve their chances of gaining a surgical training post. In addition, 61% found the level of the MCQs within the ESSQ programme to be more difficult than those in the MRCS examination. On average, year 1 students felt that they needed to spend 10 ± 6 hours per week engaged in ESSQ activities to keep up-to-date with the timetable. Respondents consistently reported that they found the ESSQ an enjoyable experience. Search of PubMed and Web of Knowledge revealed that there have been 26 full research papers and 27 abstracts published to date relating to ESSQ research dissertations submitted May 2010 onwards.
The ESSQ/MSc in Surgical Sciences was established in 2007 as a response to changes in employment law in Europe, which reduced clinical exposure in the workplace.1 Contemporaneously, changes in surgical training were implemented with the aim of streamlining progression through training milestones and shortening the duration of training. The entire ESSQ programme, based on the MRCS curriculum, is taught through self-directed learning at a distance. It is delivered via a virtual learning environment which, allows students easy access to internal and external coursework, on an “anytime, anywhere” basis via the Internet on computer or mobile devices. The learning style provides students the options of a problem-oriented entry or systematic learning illustrated through subsequent problem solving, delivered in modules scheduled in blocks of time that accommodate and complement the student's clinical training demands.
Since its inception, the ESSQ has consistently exceeded its recruitment target with the highest student numbers of any postgraduate programme at the University of Edinburgh. Although targeted specifically at surgical trainees based in the United Kingdom, it has attracted more than 500 students from 40 countries in the last 6 years. The programme appears, therefore, to have a growing international reach with applicability to the early years of surgical training in many countries. This may be related to the current general reduction in working experience for doctors; the requirement to reduce working hours in Europe, North America, and Australasia1–3 has diminished considerably the trainee's exposure to both common and less frequently encountered surgical conditions.4,5 There is, therefore, a need to consolidate the applied knowledge that may be compromised by reduced exposure to the assessment of the patient in both the elective and emergency setting.6,7 We view distance-learning surgical educational programmes as an approach different from and complementary to traditional vocational methods, and which serve to enhance the learning experience of surgeons in training, not to replace but to complement experience in the workplace. The high recruitment to the programme and positive feedback from students would endorse this view.
The design of the ESSQ programme is underpinned by the pedagogical principles of e-learning, whereby students are supported as members of an online community with clearly defined learning objectives and use of materials that are active, engaging, collaborative, and relevant.8,9 Ease of navigation around the virtual learning environment and regular support from the programme team are critical in achieving successful online learning.10 The e-learning platform developed in-house (eeSURG, http://learning.essq.ed.ac.uk) allows students to navigate resources easily, follow the programme curriculum, access course material, and interact with course tutors and other students. Central to course design, Blanchard and Frasson11 emphasize the importance of roleplay to enhance constructivist learning. The activity model within the eeSURG platform is provided by Labyrinth (http://labyrinth.mvm.ed.ac.uk), a tool for authoring and delivering case narratives. Labyrinth was designed in-house with the goal of supporting development of richly engaging, narrative medical cases that invite users to take control of their own decisions—and, by extension, their own learning—and develop the critical analysis skills to face effectively the consequences of those decisions.12 It is noted that there was an appreciable “dropout” rate from the programme each year (10%–15%) but this was less than that observed in other distance learning masters programmes; online courses are associated with higher attrition rates than on-campus equivalents,13 with several studies reporting dropout rates for online masters programmes exceeding 40%.14
Although the online ESSQ/MSc in Surgical Sciences is designed to be studied part-time, alongside full-time clinical training in a hospital setting, the absence of a significant association between students' training level and academic performance on the ESSQ programme suggests that the content of the programme stands alone and is not necessarily dependent on previous surgical experience. Greater experience held by more advanced trainees may be offset by a longer period away from formal academic study, conferring an advantage to recent graduates who are more accustomed to intensive study. Furthermore, it is recognized that each student within a particular level of foundation (intern) or surgical training can have a very different profile of skills and experience compared with others at the same level, accounting for the lack of a relationship between performance and level at entry to the programme.
Given that one of the principal aims of the ESSQ programme was to prepare the surgical trainee better for the individual components of the MRCS examination that were covered in the first 2 years to Diploma level, monitoring pass rates for those students who have presented themselves for MRCS examination in the United Kingdom has provided a valuable assessment of outcome. The increased average pass rate for ESSQ students (∼15% higher) than non-ESSQ students add credence to the positive appraisal of the programme to date and will contribute to future student engagement and motivation. The correlation between performance in the ESSQ programme and MRCS examination scores and the positive feedback from students further demonstrates that the programme is fit-for-purpose. It is accepted that a potential limitation of the current analysis is that the ESSQ may not improve MRCS outcomes per se, but rather the ESSQ students are a self-selected group of highly motivated students who were more likely to be successful in the MRCS.
The programme, however, was also intended to support the academic development of the surgical trainee at an early stage, and it is noteworthy that 116 students who enrolled in the programme since its launch in 2007 continued to complete a postgraduate degree. This has provided them with a strong academic foundation irrespective of their final career intentions. It may encourage some to pursue an additional period of research training, whereas for others, the academic exposure may replace the traditional approach, which has required surgical trainees in the United Kingdom to pursue 2 or 3 years out of surgical programme training to secure a postgraduate qualification that may make them competitive to progress thought-advanced surgical training. Although, at the present time, there is insufficient follow-up of MSc postgraduates to determine whether their participation will impact on their future surgical training plans or academic development, it is encouraging to note that 26 full research articles have been published directly from content relating to the MSc dissertation project.
There is considerable potential for distance learning programmes to have a major impact on the future delivery of surgical training. The data from this study are relatively immature to determine whether it has contributed to a major change, but it has been demonstrated that the online ESSQ programme supports the academic development of trainees in the early years of surgical training, is well received by students, and is associated with improved success in their professional examination.
The authors thank the surgical trainees for their insightful feedback on the ESSQ programme, those students who gave permission to use their MRCS examination results in this study, and Kaisey Murphy and members of the Examinations team at the RCSEd for kindly sourcing the relevant MRCS data. They thank the many e-tutors involved, both past and present, for their vital contributions to the online activities and assessments of the ESSQ programme.
1. Great Britain Department of Health. Protecting Staff; Delivering Services: Implementing the European Working Time Directive for Doctors in Training. London, UK: Great Britain Department of Health; 2003:1–3.
2. Moonesinghe SR, Lowery J, Shahi N, et al. Impact of reduction in working hours for doctors in training on postgraduate medical education and patients' outcomes: systematic review. BMJ. 2011;342:d1580.
3. Gough IR. The impact of reduced working hours on surgical training in Australia and New Zealand. The Surgeon. 2011;9:S8–S9.
4. Kairys JC, McGuire K, Crawford AG, et al. Cumulative operative experience is decreasing during general surgery residency: a worrisome trend for surgical trainees? J Am Coll Surg. 2008;206:804–811.
5. Watson DR, Flesher TD, Ruiz O, et al. Impact of the 80-hour workweek on surgical case exposure within a general surgery residency program. J Surg Educ. 2010;67:283–289.
6. Blencowe NS, Parsons BA, Hollowood AD. Effects of changing work patterns on general surgical training over the last decade. Postgrad Med J. 2011;87:795–799.
7. Canter R. Impact of reduced working time on surgical training in the United Kingdom and Ireland. Surgeon. 2011;9:S6–S7.
8. Salmon G. Flying not flapping: a strategic framework for e-learning and pedagogical innovation in higher education institutions. ALT-J Res Learn Technol. 2005;13:201–218.
10. Childs S, Blenkinsopp E, Hall A, et al. Effective e-learning for health professionals and students—barriers and their solutions. A systematic review of the literature—findings from the HeXL project. Health Info Libr J. 2005;22:20–32.
11. Blanchard E, Frasson C. An Autonomy-Oriented System Design for Enhancement of Learner's Motivation in e-Learning: International Conference on Intelligent Tutoring Systems (ITS2004), LNCS 3220
. Maceio, Brazil: Springer-Verlag; 2004:34–44.
12. Begg M. Leveraging game informed learning in higher education. Medi Teach. 2008;30:155–158.
13. Tyler-Smith K. Early attrition among first time eLearners: a review of factors that contribute to drop-out, withdrawal and non-completion rates of adult learners undertaking eLearning programmes. J Online Learn Teach. 2006;2. Available at http://jolt.merlot.org/Vol2_No2_TylerSmith.htm
. Accessed January 6, 2013.
I. Popescu (Bucharest, Romania):
First of all thank you very much for the honor of reviewing this paper. Second, congratulations for the nice work in a very important area; that of education. As a comment, in this paper, the authors have explored the impact of a surgical sciences e-learning program in supporting the academic development of surgical trainees for their professional examination. Thus, it has showed that an e-learning platform is associated with improved outcome in professional examination, master in surgical sciences. Students also had significantly higher MRCS pass rates than nonenrolled trainees; however, it is not clear how the surgical trainees were recruited in the program. Did they voluntarily enroll in the program and did they have to pass a prior examination? One might speculate that surgical trainees who were voluntarily enrolled (with or without a prior examination) could be a priori selected students (the most motivated ones and with the best performances of their generation).
Taking into consideration the aforementioned factors, and including the fact that the questions were similar to the MRCS questions, that may bias the results along with the fact that master in surgical science students have gained added knowledge and expertise within the program. So, in my opinion, it is not quite a surprise that those students have had higher MRCS pass scores.
So, finally a question, I would like to ask the authors if there was any significant differences at MRCS pass scores between the surgical trainees that graduated, the highest level with master in surgical science degree, and the other types of enrolled students. Thank you very much.
Response from O.J. Garden (Edinburgh, UK):
Thank you. You make a number of important points and the first is in terms of which trainees were allowed to enter the program. Clearly, they must have a medical degree, normally be within 5 years of graduation and be based in a supportive training environment. We do accept your point that the format of part of the assessment of our program is similar to the MCQ component of the MRCS examination. Inevitably, this is going to prepare the student better for that examination. I guess that is part of the aim, but we would also like to think that this also results from their engagement with the program over a prolonged period of time through year 1 and year 2. We have not looked specifically at the performance of the students who have gone on to complete the 3 years of the program. I think the numbers are too small, but I suspect that we will find that these are actually top performing students and I would be surprised if they did not perform well in the MRCS examination.
R.P. O'connell (Dublin, Ireland):
James, this is greatly to be welcomed because any investment in the education of our trainees, such as you have done, is greatly welcomed. But following the last question, it is a bit of a self-fulfilling prophecy that if you invest in the education, and your endpoint is how they do in the membership examination, clearly they are going to do better. My question for you is how do you know they are better clinical scientists as was the objective because it is no longer fashionable to do clinical or laboratory research? What you want to do is to produce clinician scientists and how are you going to measure that outcome?
Response from O.J. Garden (Edinburgh, UK):
I think it is too early in the program to actually assess that parameter specifically. You will appreciate that it is a contentious issue as to whether or not we should be discouraging some trainees from taking time out to go through the traditional route of academic development, and I know that our own trainees in the southeast of Scotland generally do take time out to undertake a period of full-time study toward an MD or a PhD but not all of them go on to become academic surgeons. I think that there needs to be a way of trying to support their academic development, without taking them out of the clinical training program while providing them with a quality-assured academic qualification. Clearly the students are voting with their feet in saying that that is what we actually want. On the contrary, some of the trainees who have been through this program have actually used this as a stepping-stone to full-time research. Only time will tell as to how the program is going to best fit the needs of the modern surgical trainee.
J.V. Lanschot (Rotterdam, The Netherlands):
Thank you very much. As I recently learned, the scope of the College in Edinburgh is really global. Some of the residents in The Netherlands have tried to participate in your programs and at the end of the day during the examinations they have to really learn what is the correct answer in the UK to some questions, and what the correct answer would have been in The Netherlands. So, they are in a split, they learn things that they should not learn at least in The Netherlands. My question is what percentage of your program is really global and independent of local knowledge and what percentage is in fact more or less useless to international students?
Response from O.J. Garden (Edinburgh, UK):
I am surprised to find out that the surgical evidence base in The Netherlands is different from the evidence base in Great Britain and Ireland, but you do make an important point. This examination, this academic qualification, was very much structured around training in Great Britain and Ireland, and we have been taken quite by surprise that there has been so much interest internationally, but most of the trainees outside of Great Britain and Ireland obviously feel that the content of the program is actually fit for their purpose. It is not delivered in a very didactic way, it is there to support their reflective learning and as we know, from this very meeting, there are many views sometimes on how a particular condition should be managed, when a particular operation should be performed. So, we like to think that what we are trying to do is to develop that academic ethos, to look more critically at the evidence that is there.
N. Senninger (Munster, Germany):
With reference to control mechanisms for your testing, I am convinced that you are going to be successful but you will have to prove it. I would be interested to see what happened, concerning examination performance, to the so-called dropouts or to the ones who were not permitted to the test. What happened to them in comparison to the ones who passed the test successfully?
Response from O.J. Garden (Edinburgh, UK):
That again is a very important area to try and pursue. The withdrawal rate is lower than many distance learning programs and the predominant reason for students withdrawing is for either financial reasons or a conscious decision to change career such that the programme becomes less relevant to them. Unfortunately, a lot of students who do drop out do not really come back to us when we are looking for information as to why they have left the program and what they have then gone on to do.