Skip Navigation LinksHome > July 2014 - Volume 260 - Issue 1 > Systematic Review of Learning Curves for Minimally Invasive...
Annals of Surgery:
doi: 10.1097/SLA.0000000000000596

Systematic Review of Learning Curves for Minimally Invasive Abdominal Surgery: A Review of the Methodology of Data Collection, Depiction of Outcomes, and Statistical Analysis

Harrysson, Iliana J. MD*; Cook, Jonathan PhD; Sirimanna, Pramudith MBBS; Feldman, Liane S. MD§; Darzi, Ara MD, FMedSci, FRCS, FRCSI, FRCSEd, FRCPSG, FACS, FCGI, FRCPE, FRCP; Aggarwal, Rajesh MD, PhD, FRCS

Free Access
Supplemental Author Material
Article Outline
Collapse Box

Author Information

*Department of Pediatrics, Stanford University, Stanford, CA

Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK

Department of Surgery & Cancer, Imperial College London, London, UK

§Department of Surgery, McGill University, Montreal, Canada

Department of Surgery, University of Pennsylvania, Philadelphia, PA.

Reprints: Iliana J. Harrysson, MD, Stanford University, Lucile Packard Children's Hospital, 725 Welch Road, Palo Alto, CA 94304. E-mail:

Disclosure: This study was supported in part by the Department of Surgery at Imperial College London, the NIHR CSA grant (R.A.; CS-2009-001), and Medical Scholars funding from Stanford University. The authors do not have any conflict of interest.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (

Collapse Box


Objective: To determine how minimally invasive surgical learning curves are assessed and define an ideal framework for this assessment.

Background: Learning curves have implications for training and adoption of new procedures and devices. In 2000, a review of the learning curve literature was done by Ramsay et al and it called for improved reporting and statistical evaluation of learning curves. Since then, a body of literature is emerging on learning curves but the presentation and analysis vary.

Methods: A systematic search was performed of MEDLINE, EMBASE, ISI Web of Science, ERIC, and the Cochrane Library from 1985 to August 2012. The inclusion criteria are minimally invasive abdominal surgery formally analyzing the learning curve and English language. 592 (11.1%) of the identified studies met the selection criteria.

Results: Time is the most commonly used proxy for the learning curve (508, 86%). Intraoperative outcomes were used in 316 (53%) of the articles, postoperative outcomes in 306 (52%), technical skills in 102 (17%), and patient-oriented outcomes in 38 (6%) articles. Over time, there was evidence of an increase in the relative amount of laparoscopic and robotic studies (P < 0.001) without statistical evidence of a change in the complexity of analysis (P = 0.121).

Conclusions: Assessment of learning curves is needed to inform surgical training and evaluate new clinical procedures. An ideal analysis would account for the degree of complexity of individual cases and the inherent differences between surgeons. There is no single proxy that best represents the success of surgery, and hence multiple outcomes should be collected.

Surgical and procedural technologies are evolving at a rapid pace. Dissemination of new techniques often requires training in new skills. Performance changes with experience over time, with rapid improvement seen during early learning, during which time poorer outcomes may be seen. For example, during the introduction of laparoscopic cholecystectomies in Western Australia, the prevalence of complications doubled.1 The “learning curve” is often used to describe this phenomenon and can be described as having 3 components: the starting point, the slope of the curve, and the plateau of the curve (Fig. 1).

Figure 1
Figure 1
Image Tools

Starting Point: Each person has individual experiences and backgrounds that combine and lead to an initial level of expertise in terms of performing a new procedure.

Slope: How fast a person learns a new task, or the slope of the curve. This slope varies by procedure, by person, and by how many and how often a person has performed a procedure.

Plateau: When the incremental change in the outcome being measured is not significant. This is usually the point at which a person is deemed experienced in that particular task.

Advances in surgery are constantly being made and surgeons are being trained on new procedures and devices from laparoscopic to robotic to single-incision surgeries. With this advancement, a body of literature is emerging on the associated learning curves. The existence of a learning curve clearly has implications for training and adoption of new procedures and devices. In addition, learning curves impact the evaluation of, and have been noted as a stumbling block to, rigorous evaluation of new procedures.2,3

A standardized method to analyze learning curves would be helpful in many aspects of surgical research and training including the following:

* Comparing the introduction of devices or changes in procedures to established ones

* Being able to correct for learning curves in randomized control trials (RCT)

* Comparing learning curves of institutions and surgeons to identify areas of improvement

* Defining plateau levels to be reached for residency requirements or privileges at hospitals

* Assessing the use of simulation-based training to decrease or shorten the learning curve

A systematic review of the health technology literature was completed over 10 years ago with a challenge to the health technology community to improve statistical analysis of learning curves.2,4 The number of articles that purport to address the learning curve has continued to increase, perhaps along with an increasing recognition of the importance of the learning curve in regard to training and the evaluation of new procedures and devices. Our aim is to update the previous review in the field and assess whether there have been any changes in approaches to analysis. We specifically aimed to identify the methods used to define, portray, and statistically analyze the learning curve in minimally invasive abdominal surgery and make recommendations for clinicians, credentialing bodies, and researchers.

Back to Top | Article Outline


This study was planned, conducted, and reported in adherence to PRISMA standards.5

Back to Top | Article Outline
Study Questions

The objectives of this review were to determine how learning curves are assessed in the surgical literature and define an ideal framework for this analysis with regard to the following:

* Which study designs are used to assess the learning curve?

* Which outcomes are used to assess the learning curve?

* How learning curves are portrayed and graphed?

* How learning curves are statistically analyzed?

Back to Top | Article Outline
Information Sources

A broad systematic search was performed of clinical databases (MEDLINE, EMBASE, ISI Web of Science, ERIC, and the Cochrane Library) from 1985 to August 21, 2012, to identify papers in minimally invasive surgery that mention a learning curve. In addition, a hand search of all abstracts from 1985 to 2012 in Surgical Endoscopy was performed as well as a review of all preprint articles in the American Journal of Surgery, British Journal of Surgery, Annals of Surgery, Archives of Surgery, World Journal of Surgery, Journal of the American College of Surgeons, Surgical Endoscopy, and Surgery. During the data collection process, the bibliography of each article was reviewed for additional articles.

Back to Top | Article Outline
Search Strategy

The search terms used were “laparoscopic surgery” or “simulation” or “Surgical Procedures, Minimally Invasive” and “learning curve” for Medline and “Laparoscopic surgery” or simulation or “minimally invasive surgery” and “learning curve” for ISI Web of Science, EMBASE, Cochrane, and ERIC.

Back to Top | Article Outline
Eligibility Criteria

The focus of this review is minimally invasive surgery, because this has been an area where new procedures have emerged and disseminated rapidly in the past 2 decades. For a more similar group of procedures, the articles were limited to minimally invasive abdominal surgery, that is, gastrointestinal, gynecological, or urological procedures, as well as simulated versions of those procedures. The article had to address the learning curve and formally analyze it by a graph, table, or statistical technique. Only English language articles were included. Reviews, letters, and comments were excluded.

Back to Top | Article Outline
Study Selection

Study selection was accomplished through screening the abstract of all articles collected using the search terms mentioned earlier. The full texts of the articles were then obtained and assessed to make sure that they met the criteria mentioned earlier. A second reviewer assessed 10% of these abstracts to ensure that the study selection process was robust. Any disagreements were discussed and resolved by consensus.

Back to Top | Article Outline
Data Items Collected

All variables were collected for all papers, these included which outcomes were used to assess the learning curve, how the learning curves were portrayed and graphed, statistically analyzed, and what parts of the curve were addressed: the intercept, slope, or plateau. A full list of variables can be found in Supplemental Table 1, available at

We reviewed robotic-assisted laparoscopic radical prostatectomy (RALP) in further detail; in addition to the standardized analysis, we extracted the specific data for the learning curve that the articles reported as well as the information about the specific technique, patient baseline data (age, body mass index, American Society of Anesthesiologists score, etc), and surgeon data.

Back to Top | Article Outline
Data Collection Process

A data-extraction checklist was generated and the relevant data abstracted from the full text of the selected articles. The data extraction was also completed in duplicate for 10% of the cases. The only discrepancies between the 2 reviewers that arose were in the classification of statistical techniques. These were resolved through discussion with a statistician.

Back to Top | Article Outline
Synthesis of Results

Data collection and analysis was performed in Microsoft Excel (Microsoft Corp., Redmond, WA), SPSS (IBM Corp., Armonk, NY), and Stata software (StataCorp LP, College Station, TX). To assess a trend over time, data were merged into 5-year groups; complexity of analysis was ordered by the most complex approach used from none, descriptive, split groups [eg, analysis of variance (ANOVA)], univariate trend (eg, χ2 test for trend), multivariate trend (eg, regression) to trend analysis which adjusted for clustering (multilevel regression). Pearson χ2 test and the χ2 test for trend was used to compare different studies and different time frames of studies. Statistical significance was accepted at the 2-sided 5% level.

Back to Top | Article Outline
Risk of Bias

Publication bias does most likely exist since articles that do not show a statistical difference in the learning curve may be less likely to be submitted for publication than those that do show a difference. Also, in the analysis of learning curves, more variables than published may have been collected but not included in the final analysis because they did not show a clear change in slope or outcome. Bias in the collection process was checked through duplicate data collection of 10% of the data.

Each article was graded using a system consisting of 6 components, each category receiving 1 point:

1. Adequate detail of study population (age, sex, and clinical presentation)

2. Adequate detail of technique

3. Adequate detail of assessment

4. Study powered according to sample size calculation

5. No prior knowledge of outcome before inclusion of patient (prospective)

6. Blinding of assessment.

Adequacy was determined by if the reviewer thought he/she would be able to reproduce the study from the information included in the methods section of the article.

Back to Top | Article Outline


Study Selection

We identified 6758 articles that were potentially relevant; of these, 6585 were identified through the search strategy and 173 were identified through hand searching selected journals, searching preprint articles, and reviewing references of the aforementioned articles. From these, we identified 592 articles that were relevant to abdominal laparoscopic, robotic, or simulated surgery and that formally addressed the learning curve of these procedures as seen in Figure 2. The full reference list can be found in Supplemental Table 2, available at

Figure 2
Figure 2
Image Tools
Back to Top | Article Outline
Study Design and Demographics

Table 1 summarizes the key characteristics of the included studies. Using the grading system, the articles on average received 3.4/6 points (range: 0–6). Most studies had adequate detail of population, technique, and design whereas very few (9, 2%) included a power calculation or used blinded assessment (49, 8%). The size of the studies varied from 6 to 43,028 cases with an average size of 677 cases and median of 115 cases (interquartile range: 54–300).

Table 1
Table 1
Image Tools

The number of articles addressing learning curves is steadily increasing every year as seen in Figure 3. The number of studies assessing simulation and robotics specifically is also increasing.

Figure 3
Figure 3
Image Tools
Back to Top | Article Outline
Learning Curve Outcomes

The studies measured on average 4.1 outcome variables with a range of 1 to 19. Procedure time is the most commonly used proxy for the learning curve (508, 86%). We grouped the other outcomes based on the type and timing of the outcome, that is, intraoperative outcomes, postoperative outcomes, intraoperative technical skills, and patient-oriented outcomes. Intraoperative outcomes were assessed 622 times in 333 (56%) articles. Postoperative outcomes were assessed in 322 (54%) articles for a total of 753 times. Most studies differentiated between intraoperative and postoperative complications, but 29 studies used both outcomes without differentiation and 8 provided no details about which type of complications were measured. Intraoperative technical skills were measured in 102 (17%) of the articles, 171 times. Finally, the least used outcome was patient oriented outcomes (49, 8%), which included amount of pain medication, time to oral intake, or quality of life. A detailed list of the outcomes can be found in Table 2.

Table 2
Table 2
Image Tools
Back to Top | Article Outline
Graphically Displaying the Learning Curve

Most articles (425, 72%) graphed the learning curve, with the most common method being a scatterplot (298, 50%); 107 (19%) of the scatterplots had a curve fit line superimposed, of which 64 (15%) had a regression line and 27 (6%) used a smoother such as moving average.

Many papers use straight lines to approximate the learning curve (12%), but authors also chose different curves to approximate the learning curve (11%). Not only are logarithmic and exponential curves used but also curves that allow for multiple inflection points [eg, joinpoint or spline regression) and those that allow for smoothing of the data (eg, locally weighted scatterplot smoothing (LO[W]ESS)), which uses multiple regression models]. The joinpoint regression model allows for different regression lines between points. The number of joinpoints required can be evaluated by considering the fit of the resultant curve. LOESS fit is similar in that the regression line allows for multiple inflections or changes in slope, but in this case it is done through local weighting of data and the generation of a separate linear regression line for each data point, which collectively produce a smoothed curve. However, it does not readily allow a statistical assessment of fit of the resultant curve.

Of the articles that used specific parts of the curve to define the learning curve (135, 23%), most articles used the plateau (85, 63%) and some described the slope or learning rate (34, 25%).

Back to Top | Article Outline
Learning Curve Statistical Analysis

Many papers used a formal statistical analysis of the learning curve (435, 73%), the most common being univariate split groups (413, 70%) such as student t test, χ2 or simple ANOVA. As an example, a study of 100 laparoscopic cholecystectomies would be split into 4 groups of 25 cases and the mean duration of the operation was compared between the groups.6

Univariate trend analysis (194, 33%), such as Pearson correlation, repeated-measures ANOVA, or Friedman test, was also quite commonly used. A regression model was used in 129 (22%) of analyses with linear regression (58, 10%) and logistic regression (37, 6%) being the most common. Least squares regression was used in 14 articles, nonlinear in 12, spline curves in 6, and polynomial in 5.

More authors are reflecting the inherent differences between surgeons in their statistical analysis and are adjusting for it. Twenty-one (4%) articles adjusted for clustering (typically at the surgeon level), mostly using generalized linear mixed models (13), but also generalized estimating equations (3), and hierarchical models (2). These models seek to reflect the expectation that surgeries performed by the same surgeon will be more similar to each other than surgeries by another surgeon. As well as having different underlying levels of performance, different surgeons will likely learn at differing rates. These factors lead to the grouping or clustering of data by surgeon to achieve accurate representation and analysis of the data. Risk adjustment to account for patient characteristics was calculated in only 5 studies.

Cumulative Sum (CUSUM) analysis was utilized in 25 (4%) of the articles. CUSUM analysis is used to analyze a dichotomous variable, where the author sets an acceptable failure rate and an unacceptable failure rate that seems appropriate for this variable. CUSUM analysis can be used as a way to monitor change, with the slope of the CUSUM curve at each procedure point either increasing for a failure or decreasing with a success. It has traditionally been used in a steady state process, where the acceptable complication rates are known and a person can monitor if the complication rate changes significantly. A learning curve using CUSUM would initially have a series of positive slopes, indicating that the failure rate of the surgery exceeds the acceptable values. The slope would then start to trend down, indicating that the surgeon is learning and the failure rate is now becoming acceptable.

Back to Top | Article Outline
Comparing Laparoscopic, Robotic, and Simulation Studies

Of the articles, 398 were studies on laparoscopic surgery, 94 on robotic, 76 on simulation (virtual reality simulation and desktop trainers), and 24 on animal models. Because of the inherent differences in technique, we compared the study size, type, and type of operator as well as the type of learning curve outcome that was collected. The results can be seen in Table 3. The types of outcomes collected were categorized into time and 4 groups to aid in the statistical analysis as discussed earlier.

Table 3
Table 3
Image Tools

The use of different study designs varied by study type; simulation and animal model studies used RCT studies more often than laparoscopic and robotic studies (P < 0.001) and were more often prospective (P < 0.001). Animal studies tended to have the smallest number of cases, with laparoscopic and robotic studies generally similar and simulation tending to be larger. Simulation and animal studies had students and trainee operators (with or without attending surgeons) more than the other 2 types of studies. All groups collected a significant amount of data during the procedure. The simulation studies collected predominantly technical skills data, whereas studies in the operative room collected both technical skills and outcomes data. Postoperative data and patient-specific data were only collected in laparoscopic and robotic studies.

Back to Top | Article Outline
Detailed Review of RALP

Of the 37 robot-assisted laparoscopic prostatectomy articles that were found in the original review, 4 were excluded because they did not address a continuous learning curve, that is, compared experienced with inexperienced operators, or the effect of prostate size on the learning curve. The mean size of the 33 studies was 297 procedures, median of 147 procedures with a range of 24 to 3744 procedures.7–39 All of the articles mentioned what type of equipment they used in the da Vinci robot. Eighteen studies used Vattikuti Institute prostatectomy technique40 with some adjustments. The mean/median ages of the patients ranged from 56 to 67 years, body mass index ranging from 18 to 36 kg/m2, preoperative prostate-specific antigen from 5.3 to 22.3 ng/mL. Most articles only performed surgery on clinically localized disease.

As compared with the other procedures, RALP articles looked at more variables and different ways of measuring the learning curve. Of the 33 articles, 29 assessed the learning curve using operation duration, 19 looked at estimated blood loss (EBL), 17 at positive surgical margins (PSM), 12 at length of hospital stay, and 8 at continence and complications each. Of the articles that looked at time, 26 found a statistically significant decrease in the time whereas 3 found a decreasing trend. There were 3 main ways that articles defined the cutoff for the learning curve. Many used a strict 3- or 4-hour operative time cutoff for proficiency. Others used a review of the literature and picked a common cut off like 25 or 30 procedures. Others like Sammon et al29 graphed the data and found the plateau and then split the data according to this plateau. The articles that defined the learning curve using time found a range from 12 to 750 procedures, with a median of 30 procedures. The postlearning curve operative time had a mean (SD) of 207 (46) minutes, ranging from 134 to 305. Some articles found a 2-stepped learning curve, for example, Jaffe et al17 found an initial plateau at 12 and then another at 189 procedures. Most articles defined operating time from skin to skin (12), others defined it as console time and some did not specify.

Of the articles that looked at EBL, 12 found a statistical decrease, 3 found no difference, and 23 found a trend in reduction. Of the 17 articles that looked at PSM, 12 found a statistically significant difference and 2 a decreasing trend whereas 3 found no difference. The articles that used PSM to define the learning curve found it to range from 30 to 1250 procedures, varying also by type of and stage of the cancer. Most of the articles defined the learning curve between 100 and 300 cases for PSM. Although only 2 of the 8 articles that assessed complications along the learning curve found that it decreased, 1 article defined the plateau around 150 procedures.14 Continence rates at various follow-up times were also used as a way to measure the learning curve and 6 of the 8 articles found that it increased as surgeons gained more experience. The mean grading score for these 33 articles was 3.2, with a range of 0 to 4.

To give an overview of the learning curve data, a composite graph of the operative time for learning curves data were graphed (Fig. 4).

Figure 4
Figure 4
Image Tools
Back to Top | Article Outline
Study Quality

Most studies described the detail of the study population, technique, and learning curve assessment adequately (93%, 92%, 88%, respectively). Fifty-seven percent had no prior knowledge of outcome before inclusion of the patients. Very few studies were powered according to sample size calculation (2%). Blinding of the assessment was mentioned in only 8% of the studies.

Back to Top | Article Outline
Changes in Outcomes Over Time

More advanced and varied techniques have been used over the last 10 years to analyze learning curves. There was no statistical evidence of an increase in complexity of analysis despite a numerical increase in the most complex analysis approach (regression methods which adjust for clustering) over that period. There was statistical evidence of an increase in laparoscopic and robotic studies from 1997 to 2012 (P < 0.001 both) though not simulation (P = 0.051) and animal (P = 0.481) studies. There was no evidence of a proportional change in the number of RCT (P = 0.335).

Back to Top | Article Outline


Learning new procedures are a constant part of surgical innovation and improvement and also training of new surgeons. With each new procedure comes a learning phase and this is increasingly being recognized in the literature as seen by the increase in number of articles over time. The outcomes used and analysis techniques are varied. The increase in articles using regression analysis and trend analysis shows increasing recognition of the complexity of the learning curve and the need to use methods that address this. Many trends that are seen in the data reflect the trends of the surgical field, including an increase in the number of articles discussing robotic surgery.

The outcomes that were analyzed often seemed to reflect the accessibility of data collection, and may not necessarily be those of greatest clinical importance. For example, many simulators collect error rate, instrument movement and path length automatically whereas these outcomes are hardly ever seen in clinical cases. Similarly, complications, blood loss, and duration of the procedure are often recorded during surgeries, and this data would be readily available for retrospective or prospective reviews. Even though these are the most commonly used outcomes, some studies do choose to look at postoperative patient outcomes including readmission, reoperation, or pain, which is important when assessing the effect of the learning curve on the system and patient as a whole.

CUSUM analysis continues to be used in learning curve analysis but the limitation remains that the investigator has to determine the acceptable failure rate for the outcome that is being studied. At this moment, there is typically insufficient data to confidently set failure rates. Once learning curves for procedures have been determined, and an agreed upon failure rate has been established, CUSUM analysis can be used to assess individual practitioners' learning. The CUSUM analysis is indeed utilized in real-time in manufacturing settings, and it could be used in the same manner for surgical performance. There would be the possibility to prompt review of the surgeon's operative outcomes, based on a predefined failure rate. In concert, the CUSUM analysis could also identify a surgeon with an accelerated learning curve, which would identify the surgeon for independent practice at an earlier stage of the predefined learning pathway.

Back to Top | Article Outline
Ideal Assessment of the Learning Curve

This review analyzed 592 articles, though only 9 (1.5%) of studies were a priori powered to calculate a statistical difference in the learning curve for specific outcomes related to the procedure of interest. There is the possibility of bias with regard to the outcomes measured and the techniques utilized for analysis of the learning curve, and the findings reported by authors. It thus makes it difficult to conclusively delineate a set of guidelines for learning curve analysis. Instead, we would like to make recommendations, not define a standard, as to how to ideally assess a learning curve, knowing that the resources available and sample size vary significantly between procedure and institution. To be able to realize the full potential of a learning curve, to use it when comparing the introduction of devices or changes in procedures, correcting for learning curves in RCTs, identifying areas of improvement for surgeons or institutions, defining levels for residency or privileges at hospitals, and assessing the use of simulation-based training, a standard approach to the analysis of the learning curve needs to be adopted. There is a great deal of complexity in defining values for an ideal learning curve as this article has attempted to elucidate. There are multiple factors that lead to a successful procedure, which include the skills and experience of the surgeon, the operative team and also the varying complexity of individual cases. An ideal analysis would include these aspects in a multivariate analysis.41 There is no single outcome that best represents the success of a procedure hence multiple outcomes are preferable and it would be reasonable to expect the impact of learning to vary between outcomes. Possible outcomes include operative time, and intraoperative and postoperative outcomes such as complications, transfusions, recurrence, and even though more time-intensive, technical skills score and patient quality of life. In this study, time was the most frequently used outcome measure to define the learning curve. Although simple to measure, this metric undermines the complexity of the operative procedure with regard to learning curve analyses. External factors such as patient and disease variability, operative team familiarity, and institutional dynamics can affect the time taken to perform an operative procedure. Furthermore, time may not be the measure of prime importance with regard to maintenance of quality and safety of the surgical intervention. Many outcomes confound the usefulness of using procedure time as an outcome variable, and if used to assess learning curves, other measures of performance and patient complexity must also be assessed. The outcomes need to be clearly defined to be meaningful.42 Because of the inherent differences between surgeons and institutions, more than 1 surgeon and more than 1 institution should be assessed and the differences accounted for in the analysis by generalized estimating equations or linear mixed models. Sequential cases of the same surgeon are preferred to cases of surgeons at different levels for these same reasons. A graphical display of the data should be provided whenever possible. In summary, an ideal learning curve assessment would include the following:

* Multivariate analysis: accounting for different surgeons, teams, and patients

* Multiple clearly defined outcome variables

* Multiple institutions and surgeons

* Graphical display of data

It is also important to note that learning in the clinical context does not occur according to a single trajectory, or line. Training is now becoming more refined, involving a stepwise process of knowledge, simulation-based practice, achievement of competence levels, and guided progression through the complexity of the operative procedure. There are statistical techniques that take this into account, such as joinpoint or spline regression, 9 (1.5%) of the articles used these statistical analyses.

Back to Top | Article Outline

This systematic review of the literature is limited by the reporting and methodology of the published work to date. Most articles did not report a power analysis for assessment of the learning curve. It is thus possible that the outcomes measured and techniques used were skewed to methods that were found to be statistically significant. The search criteria, even though broad, could have missed articles. A total of 40 articles were found through hand searching journals and reviewing references that had been missed in the initial search that found 6585 articles. The articles were mostly reviewed and data extracted by 1 reviewer, which could have led to reviewing inconsistencies. A second reviewer did review 10% of the articles to ensure that the data collection followed the determined protocol. The level of reported data and procedures made it difficult to summarize across studies and limited the analysis.

Back to Top | Article Outline

This review is a comprehensive review of the surgical learning curve literature to date, reviewing 592 articles, which is larger than any previous review.2 We discuss the state of learning curve analysis with recommendations that can be used by authors who wish to assess the learning curve.

Back to Top | Article Outline

The learning curve is expensive and can lead to morbidity and mortality. At a population level, there is concern that it could lead to decreased quality of care and increased costs of care. This leads to a decreased value of health care during the learning period (value being quality divided by costs).43 By understanding the learning curve better, we would be able to start recouping the value of innovation and learning, an area yet to be investigated.

Back to Top | Article Outline


Learning curves are an important aspect of complex procedures. Assessment is essential to inform surgical training and evaluation of new procedures in the clinical setting. Learning curves are being analyzed in a variety of ways. By reviewing the existing learning curve literature, we have developed basic guidelines for choosing the outcome to be analyzed, statistical analysis, and graphical display of the learning curve. There is still a need for further improvement in the methodology and data used to provide more informative findings though some examples of large and informative studies have been conducted.44,45

Back to Top | Article Outline

The author contributions were as follows: Study concept and design: Harrysson, Aggarwal, Cook, Feldman. Analysis and interpretation of data: Harrysson, Aggarwal, Cook, Sirimanna. Drafting of Manuscript: Harrysson, Aggarwal, Cook. Critical revision of the manuscript: Harrysson, Aggarwal, Cook, Feldman, Darzi. Statistical analysis: Cook. Iliana Harrysson has full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Back to Top | Article Outline


1. Hobbs MS, Mai Q, Knuiman MW, et al. Surgeon experience and trends in intraoperative complications in laparoscopic cholecystectomy. Brit J Surg. 2006;93:844–853.

2. Ramsay CR, Grant AM, Wallace SA, et al. Assessment of the learning curve in health technologies: a systematic review. Int J Technol Assess Health Care. 2000;16:1095–1108.

3. Cook J, Ramsay C, Fayers P. Statistical evaluation of learning curve effects in surgical trials. Clin Trials. 2004;1:421–427.

4. Ramsay CR, Wallace SA, Garthwaite PH, et al. Assessing the learning curve effect in health technologies: lessons from the nonclinical literature. Int J Technol Assess Health Care. 2002;18:1–10.

5. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med. 2009;151:264–269.

6. Joseph M, Phillips M, Rupp CC. Single-incision laparoscopic cholecystectomy: a combined analysis of resident and attending learning curves at a single institution. Am Surgeon. 2012;78:119–124.

7. Ahlering TE, Skarecky D, Lee D, et al. Successful transfer of open surgical skills to a laparoscopic environment using a robotic interface: initial experience with laparoscopic radical prostatectomy. J Urol. 2003;170:1738–1741.

8. Artibani W, Fracalanza S, Cavalleri S, et al. Learning curve and preliminary experience with da Vinci-assisted laparoscopic radical prostatectomy. Urol Int. 2008;80:237–244.

9. Atug F, Castle EP, Srivastav SK, et al. Positive surgical margins in robotic-assisted radical prostatectomy: impact of learning curve on oncologic outcomes. Eur Urol. 2006;49:866–872.

10. Bhandari A, McIntire L, Kaul SA, et al. Perioperative complications of robotic radical prostatectomy after the learning curve. J Urol. 2005;174:915–918.

11. Burgess SV, Atug F, Castle EP, et al. Cost analysis of radical retropubic, perineal, and robotic prostatectomy. J Endourol. 2006;20:827–830.

12. Chin JL, Luke PP, Pautter SE. Initial experience with robotic-assisted laparoscopic radical prostatectomy in the Canadian health care system. Can Urol Assoc J. 2007;1:97–101.

13. Dev H, Sharma NL, Dawson SN, et al. Detailed analysis of operating time learning curves in robotic prostatectomy by a novice surgeon. BJU Int. 2012;109:1074–1080.

14. Doumerc N, Yuen C, Savdie R, et al. Should experienced open prostatic surgeons convert to robotic surgery? The real learning curve for one surgeon over 3 years. BJU Int. 2010;106:378–384.

15. Gumus E, Boylu U, Turan T, et al. The learning curve of robot-assisted radical prostatectomy. J Endourol. 2011;25:1633–1637.

16. Hong YM, Sutherland DE, Linder B, et al. “Learning curve” may not be enough: assessing the oncological experience curve for robotic radical prostatectomy. J Endourol. 2010;24:473–477.

17. Jaffe J, Castellucci S, Cathelineau X, et al. Robot-assisted laparoscopic prostatectomy: a single-institutions learning curve. Urology. 2009;73:127–133.

18. Ko YH, Ban JH, Kang SH, et al. Does robot-assisted laparoscopic radical prostatectomy enable to obtain adequate oncological and functional outcomes during the learning curve? From the Korean experience. Asian J Androl. 2009;11:167–175.

19. Lebeau T, Roupret M, Ferhi K, et al. The role of a well-trained team on the early learning curve of robot-assisted laparoscopic procedures: the example of radical prostatectomy. Int J Med Robot Comp. 2012;8:67–72.

20. Leroy TJ, Thiel DD, Duchene DA, et al. Safety and peri-operative outcomes during learning curve of robot-assisted laparoscopic prostatectomy: a multi-institutional study of fellowship-trained robotic surgeons versus experienced open radical prostatectomy surgeons incorporating robot-assisted laparoscopic prostatectomy. J Endourol. 2010;24:1665–1669.

21. Mayer EK, Winkler MH, Aggarwal R, et al. Robotic prostatectomy: the first UK experience. Int J Med Robot. 2006;2:321–328.

22. Ou YC, Yang CR, Wang J, et al. Robotic-assisted laparoscopic radical prostatectomy: learning curve of first 100 cases. Int J Urol. 2010;17:635–640.

23. Ou YC, Yang CR, Wang J, et al. Learning curve of robotic-assisted radical prostatectomy with 60 initial cases by a single surgeon. Asian J Surg. 2011;34:74–80.

24. Ou YC, Yang CR, Wang J, et al. The learning curve for reducing complications of robotic-assisted laparoscopic radical prostatectomy by a single surgeon. BJU Int. 2011;108:420–425.

25. Philippou P, Waine E, Rowe E. Robot-assisted laparoscopic prostatectomy versus open: comparison of the learning curve of a single surgeon. J Endourol. 2012;26:1002–1008.

26. Ploussard G, Xylinas E, Salomon L, et al. Robot-assisted extraperitoneal laparoscopic radical prostatectomy: experience in a high-volume laparoscopy reference centre. BJU Int. 2010;105:1155–1160.

27. Raman J, Dong S, Levinson A, et al. Robotic radical prostatectomy: operative technique, outcomes, and learning curve. JSLS. 2007;11:1–7.

28. Samadi D, Levinson A, Hakimi A, et al. From proficiency to expert, when does the learning curve for robotic-assisted prostatectomies plateau? The Columbia University experience. World J Urol. 2007;25:105–110.

29. Sammon JPA, Beaule L, Kinkead T, et al. Robot-assisted radical prostatectomy: learning rate analysis as an objective measure of the acquisition of surgical skill. BJU Int. 2010;106:855–860.

30. Schroeck FR, de Sousa CA, Kalman RA, et al. Trainees do not negatively impact the institutional learning curve for robotic prostatectomy as characterized by operative time, estimated blood loss, and positive surgical margin rate. Urology. 2008;71:597–601.

31. Sharma NL, Papadopoulos A, Lee D, et al. First 500 cases of robotic-assisted laparoscopic radical prostatectomy from a single UK centre: learning curves of two surgeons. BJU Int. 2011;108:739–747.

32. Sim HG, Yip SK, Lau WK, et al. Team-based approach reduces learning curve in robot-assisted laparoscopic radical prostatectomy. Int J Urol. 2006;13:560–564.

33. Sooriakumaran P, John M, Wiklund P, et al. Learning curve for robotic assisted laparoscopic prostatectomy: a multi-institutional study of 3794 patients. Minerva Urol Nerol. 2011;63:191–198.

34. Thiel DD, Francis P, Heckman MG, et al. Prospective evaluation of factors affecting operating time in a residency/fellowship training program incorporating robot-assisted laparoscopic prostatectomy. J Endourol. 2008;22:1331–1338.

35. Tsao AK, Smaldone MD, Averch TD, et al. Robot-assisted laparoscopic prostatectomy: the first 100 patients-improving patient safety and outcomes. J Endourol. 2009;23:481–484.

36. Tsivian M, Zilberman DE, Ferrandino MN, et al. Apical surgical margins status in robot-assisted laparoscopic radical prostatectomy does not depend on disease characteristics. J Endourol. 2012;26:361–365.

37. Wu ST, Tsui KH, Tang SH, et al. Laparoscopic radical prostatectomy: initial experience of robotic surgery in Taiwan. Anticancer Res. 2008;28(4A):1989–1992.

38. Zorn KC, Orvieto MA, Gong EM, et al. Robotic radical prostatectomy learning curve of a fellowship-trained laparoscopic surgeon. J Endourol. 2007;21:441–447.

39. Zorn KC, Wille MA, Thong AE, et al. Continued improvement of perioperative, pathological and continence outcomes during 700 robot-assisted radical prostatectomies. Can J Urol. 2009;16:4742–4749.

40. Menon M, Tewari A, Peabody J, et al. Vattikuti Institute prostatectomy: technique. J Urol. 2003;169:2289–2292.

41. Xu R, Carty MJ, Orgill DP, et al. The teaming curve: a longitudinal study of the influence of surgical team familiarity on operative time. Ann Surg. 2013;258:953–957.

42. Cook J, Ramsay C, Fayers P. Using the literature to quantify the learning curve: a case study. Int J Technol Assess Health Care. 2007;3:255–260.

43. Lee TH. Putting the value framework to work. N Engl J Med. 2010;363:2481–2483.

44. Vickers AJ, Savage CJ, Hruza M, et al. The surgical learning curve for laparoscopic radical prostatectomy: a retrospective cohort study. Lancet Oncol. 2009;10:475–480.

45. Flum DR, Koepsell T, Heagerty P, et al. The nationwide frequency of major adverse outcomes in antireflux surgery and the role of surgeon experience, 1992–1997. J Am Coll Surg. 2002;195:611–618.

learning curve; minimally invasive surgery; surgical education

Supplemental Digital Content

Back to Top | Article Outline

© 2014 by Lippincott Williams & Wilkins.


Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.