The field of surgery has a unique culture and rich tradition of innovation.1 Surgeons are trained to perform continuous situational assessment, decision analysis, and improvisation, in preparation for the variations in anatomy, physiology, and human performance that define the practice. Indeed, as Riskin et al1 pointed out, “most surgeons innovate on a daily basis, tailoring therapies and operations to the intrinsic uniqueness of every patient and their disease.”
In contrast to the introduction of a new drug or medical device, the modification of a standard surgical technique is not generally subjected to formal oversight or monitoring of outcomes. The surgeon at work is free to tinker with procedures, introducing modifications of varying degrees to the point where a procedure could arguably be called “new.” Rapidly advancing technology offers a steady stream of opportunities to innovate, and there is incentive to do so to improve quality of care, attract new patient referrals, or advance academically. But innovation can become problematic when patients may be subjected to procedures that are more harmful, more costly, or less efficacious than the “old way.”
In August 2004, the director of the Association of American Medical Colleges Council of Academic Societies Affairs sent a memo to the Presidents of 5 major surgical societies, suggesting that leaders of the surgical community should provide guidance on the issue of surgical innovations. The Society of University Surgeons (SUS) organized a project team to clarify the distinction between “variations,” “innovations,” and “research,” and to offer guidelines to surgeons and hospitals for appropriate implementation and oversight of surgical innovations. The team's findings were presented and discussed in an open forum at the Academic Surgical Congress in February 2007; on the basis of discussion and feedback, the guidelines were finalized and a Position Statement was published in June 2008.2 It was recommended that institutions create surgical innovation committees (SICs) to ensure appropriate oversight and disclosure of innovations, and that a national registry of innovations be created. At the time of publication, the American College of Surgeons had expressed willingness to assist with the creation and management of a national registry of surgical innovations. Unfortunately, this unfunded project did not continue—yet surgical practice has continued to evolve, and perhaps more rapidly than ever.
The purpose of this study was to determine the level of awareness of the SUS Position Statement among leaders of academic departments of surgery more than 4 years after the publication of the Statement. The study was designed to assess whether SICs exist and if so, how they function.
A survey to assess the awareness and implementation of the SUS Project Team's recommendations regarding SICs was developed and administered using the Web-based tool SurveyMonkey. The survey questions were designed with 3 goals: to assess awareness of the Position Statement recommendations; to assess whether the respondent's institution had an SIC and if so, what were the characteristics of that SIC or if not, why not; and to collect general demographics on the respondent's institution. The survey was also designed to be dynamic, with the answer to one question directing the respondent to other specific questions. The final survey questions are included in the Appendix. The survey software was set to disallow more than a single survey response from the same Internet Protocol address, which would preclude multiple responses from within the same academic institution (or Internet hosting network). The survey questions allowed respondents to give only 1 response, unless otherwise indicated (eg, “check all that apply”).
A brief protocol describing the survey project, including the survey questions and the “cover letter” to potential participants, was submitted to the institutional review board (IRB) of Albany College of Pharmacy and Health Sciences. The IRB determined that the protocol was exempt from IRB review.
The September 2012 edition of the public directory for the Society of Surgical Chairs, a membership organization, which includes the Chairs of all academic Departments of Surgery in the United States and Canada, was used to create an electronic mailing list. The electronic mailing list included 157 e-mail addresses, including 144 Chairs at US institutions and 13 Chairs at Canadian institutions. The survey was distributed electronically 3 times between November 26, 2012, and January 2, 2013, with a cover letter describing the project, asking for the recipient's participation, and containing a link to the online survey. The survey link was closed on January 11, 2013, and the data were extracted and analyzed.
The data were extracted from the SurveyMonkey software into Microsoft Excel. Descriptive statistics, where reported, were performed by the author (McNair).
A total of 65 respondents entered “yes” to the initial question asking their agreement to participate and began the survey; however, respondents could choose not to answer any question, so the total number of responses for any specific question may be less than 65. Percentages of responses are calculated on the basis of the total number of responses for each question. All comments that were entered as free text (eg, in the free text “specify” box if the response chosen was “other”) are reproduced here exactly as they were entered in the survey.
Survey Response Rate
From the initial mailing list of 157 e-mail addresses, 5 e-mail addresses appeared to be incorrect (ie, the e-mail was returned to sender with an error message denoting it as undeliverable). One e-mail address returned an automatic response that the intended recipient did not read e-mail, and one returned an automatic response that the respondent was on leave of absence and not reading messages. Some e-mails replied with “out of office” messages on 1 or 2 but not all 3 of the mailings; these are considered successful mailings. Therefore, it is estimated that 150 of the 157 attempted e-mail addresses should have reached the intended recipient on at least 1 of the 3 attempts. The overall response rate to the survey was 65 responses out of 150 presumed successful contacts, giving a 43% response rate.
Demographics of Respondents and Their Institutions
The demographics of the survey respondents are summarized in Table 1. The majority of respondents (59/62, 95%) indicated that they were in fact the Chair of the Surgery Department at their institution, whereas 3 indicated that they were not the Chair but were a member of the surgical faculty. The surgical specialties of the respondents were varied across a large number of surgical fields, with general surgery and trauma/acute care surgery the most frequent responses (15/62, 24% each). The next most frequent responses were vascular surgery (9/62, 15%), surgical oncology and other (not defined) specialty (5/62, 8% each), and minimally invasive surgery (4/62, 7%). Four additional respondents indicated that they had a specialty other than surgery, but when asked to describe in free text, all 4 reported fields that are generally considered to be surgical subspecialties (“Urology,” “Orthopedics_Pediatric,” “Orthopedic,” and “also do general surgery”).
Information was also collected regarding the characteristics of the institutions at which the respondents were based, as summarized in Table 2. A majority of respondents (81%) indicated that they were from centers with more than 400 beds, and more than half of the respondent's institutions (55%) perform more than 25,000 surgical procedures annually. Consistent with the targeted survey group, 95% confirmed having a surgical residency program based at their institution.
Respondents were also asked whether surgical innovation was promoted or considered to be one of the strengths of their facility. A large majority of survey respondents (84%) reported that it was. Because a great deal of surgical innovation over the last 2 decades has been in the field of minimally invasive surgery (including laparoscopic and endoscopic procedures replacing traditional open surgical procedures), the survey also asked whether minimally invasive surgery was considered to be a strength of the institution. The vast majority of respondents (98%) indicated that it was.
Awareness of Recommendations and Presence of an SIC
Awareness of the SUS recommendations and whether the institution had an SIC was also reported (Table 3). The majority of respondents (86%) indicated that within their institution or department, they had discussed the distinction between surgical practice, innovation and research. Of these 55 respondents, 49% reported that these had been formal discussions or meetings whereas 51% reported that the discussions had been informal between colleagues. Fourteen percent (9/64) of respondents reported that there had been no discussions of this topic at their institution; 5 of these 9 respondents were at institutions that promote innovative surgery. More than half the respondents (55%) indicated that they were unaware of the 2008 Position Statement written by the SUS Task Force; 22% (14/64) of respondents had heard the paper mentioned or discussed but had not read it.
Overall, 23% (14/61) of respondents report that their institution has an SIC (or committee with the responsibilities of an SIC), whereas 20% (12/61) report that they have discussed and SIC or are planning to put one in place. The largest proportion of respondents, 38% (23/61), said that they have never considered an SIC, and another 20% (12/61) said that they have considered an SIC but do not see the need for one.
All respondents, regardless of whether they reported an SIC, were asked about the infrastructure that oversaw innovative surgery at their institution. The most frequent response was morbidity and mortality conferences (“M&M”) (88%). Other frequent responses included formal peer review (77%), professionalism and integrity of the surgical staff (67%), and informal peer review (49%), with 51% also reporting the use of an outcomes registry.
Additional analyses of the data were done to further explore the subgroups of respondents and their institutions, particularly with regard to whether an SIC was present at the institution, was discussed or planned, had been decided against, or had never been discussed. For respondents, the status of an SIC was examined with regard to whether or not they had internal discussions about surgical practice, innovation, and research (Table 4). There seemed to be a relationship between awareness of the Position Statement, formal discussions about surgical innovation and research, and the formation of an SIC. Conversely, unawareness of the Position Statement was related to the nonconsideration of forming an SIC.
The status of the SIC was also examined with regard to characteristics of the institution including the number of surgical procedures, and whether innovative surgery or minimally invasive surgery were promoted as a strength of the institution (Table 5). There did appear to be a relationship between the surgical volume and consideration of an SIC, with 71% of the centers which did less than 10,000 procedures a year reported that they had never considered an SIC, compared with 52% of the centers doing 10,000 to 25,000 procedures a year, and only 21% of the highest volume centers (more than 25,000 procedures a year) reporting that an SIC had never been considered. None of the lowest-volume centers reported having an SIC, whereas 14% of the mid-volume and 32% of the high-volume centers had an SIC. Of the centers that reported that innovative surgery was promoted as an institutional strength, 27% had an SIC whereas 31% had never considered it; of those centers which did not promote innovative surgery, 30% had discussed an SIC but none had one.
Characteristics of the SIC at Institutions With an SIC
All respondents who reported that their institution had an SIC also reported that the institution promoted both innovative surgery and minimally invasive surgery. Respondents who reported that there was an SIC at their institution were asked about the characteristics of the SIC, with the responses described in Table 6.
The number of members on the SIC ranged from 3 to 40. The mean number of members was 11, the median number was 7, and the mode was 6. Most committees included at least 1 faculty surgeon (85%), a department chair other than surgery (77%), and/or the surgery department chair (69%). Patient safety specialists (39%) or patient advocates (15%) were included on some committees. Only 31% of committees included an ethicist or ethics committee member.
Ten SICs met on a regular scheduled basis: 6 monthly, 3 less frequently, and 1 more frequently. Four reported meeting on an as-needed basis, including one that was rolled into a separate meeting that occurred more frequently than monthly.
Overall, the existing SICs seem to review few innovative procedures. Although the majority of institutions represented are large institutions with a high surgical volume, only 1 SIC reviewed more than 20 procedures within the last year. Thirty-eight percent of SICs reviewed only 4 to 10 procedures in the prior year, whereas 50% (7/14) reviewed 1 to 3 procedures and 1 SIC reviewed none.
There was no clear relationship between the size of the SIC, the number of reviews performed within the last year, the meeting frequency, or the number of procedures performed annually at the institution; for example, the number of reviews was not higher at high-volume centers, nor was the meeting frequency. Characteristics of each of the reported SICs are described in Table 7.
Institutions Without an SIC
Of the 12 respondents who indicated that they had considered forming an SIC at their institution but did not see the need for one, 11 provided further information about this decision (Table 8). One indicated that no innovative surgery was performed at their institution; this respondent also indicated that innovative surgery was promoted as strength of the institution, which seems to have been an inconsistency. The most common response (73%) was that there was an existing infrastructure that served this purpose. Almost half (46%) of the institutions without an SIC responded that the IRB oversees innovations at their institution, whereas 37% reported oversight by their ethics committee. Four respondents (36%) also reported that this kind of activity should be guided by the professional conduct of the surgeon. Two respondents wrote in “other” responses indicating “dont [sic] know—new at institution” and “no time to proceed with organizing.”
Demographics of Survey Respondents
The survey described herein was distributed to academic surgical Chairs throughout the United States and Canada. This subgroup was selected because they are leaders in institutions where there is likely to be a residency training program, a broad scope of surgical practitioners, a connection including some oversight of those practitioners, and opportunities for innovation. The response rate of 43% is considered reasonably good for such a study. The responses indicated that the respondents came from generally large institutions, with 81% from institutions with more than 400 beds. By comparison, in a 2010 survey of 338,607 United States hospitals, just 49% of hospitals had more than 400 beds.3 Thus, although this might suggest the results are not representative of the broader sample of hospitals in the United States, the demographics of both the respondents and the institutions that they represent are consistent with the characteristics of larger health care centers, which have academic surgery departments and residency training programs in surgery. Surgical volume was also generally high, with just more than half of the institutions performing more than 25,000 surgical procedures a year, and 84% of respondents indicated that their institution promotes and advertises their facilities as centers for innovative care and innovative procedures.
Awareness of SUS Position Statement
Only 23% of the responding Chairs of surgery departments indicated that they had read the SUS Position Statement on innovations2; the majority (55%) were completely unaware of the document. The fact that 86% reported having formal (42%) or informal (44%) discussions within the institution or department about surgical practice, innovation and research distinctions, may reflect the initiative of other surgeons who are interested in innovations or surgical ethics or who were familiar with the Position Statement. Regardless, and despite the large number of respondents indicating that their institution promotes innovative surgery, only 23% of report that their institutions have an SIC. Because the majority of Chairs are unaware of the recommendations, this is not a surprising finding. Of course, there are others within an institution who may advocate for or drive the establishment of an SIC (such as risk managers or the institution's IRB).
More fundamentally, however, these findings highlight the apparent confusion that persists regarding innovative surgery and appropriate oversight. It is possible that an institution promoting “innovative surgery” is actually marketing “cutting edge” techniques such as minimally invasive thoracic surgery, endovascular surgery, or robotic surgery and the word “innovative” is chosen for marketing value rather than for accuracy. A Department Chair unfamiliar with the SUS Position Statement may not recognize the distinction. This could explain seemingly discrepant responses such as those reported in Table 4. For example, 1 of the 9 respondents who reported that this topic had never been discussed, reports having an institutional SIC. Another 6 were unaware of the paper, yet claimed to have SICs at their institution. Perhaps they misunderstood the rationale or role of an SIC. Indeed, the largest group of respondents (38%) had never considered having an SIC. Another 20% had considered forming an SIC but did not see the need for one at their institution, and when asked to elaborate 5 of the 12 respondents stated that their IRB reviewed this type of procedure, and 4 stated that the hospital ethics committee review this type of procedure (2 cited both the IRB and ethics committee as responsible for review).
It is possible that institutional policies extend the responsibility of the IRB to the review of innovative procedures which do not meet the formal definition of research, although this would be an unusual arrangement and outside the scope of the usual functions of an IRB, which generally focus on review of projects that meet the regulatory definitions of human subject research. An assumption that the institution relies on the IRB for review of innovative procedures that do not meet the regulatory definition of human subject research may indicate confusion about an IRB's role and responsibility and a lack of education regarding the definition of research that would fall under the review of an IRB. This finding is consistent with published research that indicates that practicing surgeons have a poor knowledge of the purpose and basic regulations of an IRB.4
In sum, the lack of familiarity with the SUS Position Statement may explain why so few Chairs reported having an SIC. The recognition of what is truly an innovation requiring oversight, versus a cutting-edge technique, is important in determining how they are managed. In both cases, it is critical that the surgeon have the appropriate training; that the patient be informed of the procedure, alternatives, and expected risks, benefits, and outcomes; and that the surgeon tracks his or her results. In the case of a new, “cutting-edge” technique, there are likely published data available to benchmark outcomes; on the other hand, if a surgeon is innovating, it is imperative that the outcomes are tracked and compared with alternative techniques. This may require an impartial SIC.
Respondents, whether or not they indicated that they had an SIC, were asked about the infrastructure to oversee innovative surgery within the institution. The most frequent response was morbidity and mortality (M&M) conferences (88%), whereas the use of an outcomes registry (documentation and recording of results after a procedure has been completed) was also a frequent response (51%). However, both of these systems provide only retrospective review after the procedure has already occurred and, in the case of the M&M conferences, review occurs only if the patient had a poor outcome or died. Although these mechanisms may be useful for the surgeons performing the procedures or planning future procedures, retrospective review cannot add any protections for the patients on whom the procedures have already been performed. The retrospective forms of review also fail to address any issues of informed consent and whether patients knew that the innovative procedure they underwent was not the standard surgical procedure. As was suggested by the SUS project team, M&M conferences may provide an opportunity for open, critical discussions of difficult patients or problems and might allow innovators to present new techniques and elicit peer commentary, but should not be viewed as a substitute for formal review.2 Looking to one's colleagues for guidance is important, but it risks potential bias. Respondents also cited formal (77%) and informal (49%) peer review, although the survey did not collect any additional detail on what these mechanisms entailed or whether they were prospective or retrospective.
Volume of Innovative Procedures Being Reviewed
A notable finding of the survey was that within the 14 institutions that had an SIC (or a functionally equivalent committee), the number of procedures reviewed within the past year was very low. Respondents indicated that innovative surgery was promoted for all of these institutions. Leaving aside the institution that reviewed no procedures, and the institution that reviewed more than 20 (since we cannot assign a numerical value to the actual number of reviews), we can make a conservative estimate of the percentage of procedures being reviewed based on the highest end of the range of procedures reviewed, and the lowest end of the range of procedures performed. With this formula, the SICs of the remaining institutions were reviewing only 0.012% to 0.1% (3/25,000–10/10,000) of the annual procedures performed. There are several potential explanations for this finding, including that in these institutions that promote innovation as an institutional strength only a very small fraction of their surgeries are actually innovative. Other possible explanations are that surgeons responding to the survey are underestimating the number of procedures under review, that the surgical faculty at these institutions are not aware of or educated about the SIC and don’t know that they should be submitting innovative procedures for review, or that innovative procedures are not being identified as such by the surgeons (which may also be an issue of education). It may also be that the SICs doesn’t know what it should be reviewing and is allowing innovative procedures to be performed without review or is directing a large number of innovative procedures to the IRB for review as research.
Limitations of the Survey
Although the survey provided useful information, there are some limitations. The response rate of 43% is generally considered to be a good response rate for an unsolicited survey; however, the possibility that a representative sample was not obtained must be considered. It is possible that recipients did not respond because they disagree with the concept of SICs. Because very few of the responding institutions have reported having an SIC, it is difficult to make any conclusion about trends or common practices in how SICs are constituted or run. Although there is a body of literature regarding how to improve the response rates to physician-directed surveys,5–7 and the necessary sample size for qualitative survey assessments can be calculated, there is no conclusive literature that provides guidance for what can be considered an adequate sample size for a qualitative survey of this type.
The survey, by design, was brief to attempt to maximize participation rates. The brevity of the questions also means that the details and implications of some responses cannot be assessed fully. Overall, the institutions with SICs indicated that a very low number of innovative procedures are going to the SIC for review, but this survey cannot assess whether that is because the volume of procedures that meet the definition of “innovative” is actually very low, or because innovative procedures are not being identified and reviewed accurately.
Finally, the lack of awareness of the original SUS Position Statement may have affected the responses. For example, unfamiliarity with the distinction between research, innovations, and surgical advances may have resulted in confusion in interpreting the questions, and/or apparent discrepancies in the responses.
The data obtained in this survey project provide interesting and previously unknown information about the state of the oversight of surgical innovation within the United States and Canada. The results of the survey demonstrate that despite the formation of a Task Force convened by the SUS and publication of recommendations in the widely read Journal of the American College of Surgeons, the majority of surgery department Chairs is unaware of these recommendations. Even at the institutions where SICs had been formed, and where the performance of innovative surgery was promoted as an institutional strength (and therefore presumed to be performed with some frequency), the SICs reviewed only a fraction of the procedures performed, which makes them of questionable value in the protection of patients or in improving the awareness and education of the surgical faculty to the ethical issues associated with surgical innovation.
Most institutions still seem to rely on the professionalism and ethical behavior of the surgical faculty, and on informal and/or retrospective mechanisms, as the primary methods of oversight of surgical innovation. However, neither these data nor the body of literature surveyed give confidence that these methods are actually providing adequate protection for patients. Concerns still exist with regard to ensuring adequate informed consent for procedures that are not considered standard, and with regard to surgeons being able to appropriately identify when they are engaged in behavior that meets the definition of research and requires IRB review. To place the burden of this identification on the surgical faculty requires an organizational culture of professionalism and ethical behavior, and well as good and consistent education about clinical research and research ethics, which may not sufficiently exist within institutions. Although the potential reasons are myriad, the conclusion must be reached that within academic institutions, patients are not being adequately protected with regard to the performance of innovative surgical procedures.
The authors thank Sean Philpott-Jones, PhD, MSB, and Lawrence McCullough, PhD, for their assistance in the review of the survey design, and their review and helpful discussion of the study results.