If you’ve been reading my blog, you know I’ve been thinking a lot about the larger implications of the mess at Duke University. For anyone who has missed it thus far, the (very) brief version is that an up-and-coming young faculty member, Anil Potti, MD, reported in several papers to have discovered gene expression signatures that could predict cancer patients’ response to treatment. Despite concerns from other researchers about the quality of the work – which has now been thoroughly discredited – the university proceeded with three clinical trials based on it, enrolling approximately 110 patients before the trials were abruptly terminated.
The fallout from the Potti problem is widespread and getting wider. However, I think one issue that is not getting the attention it deserves is that of institutional conflict of interest.
The Cancer Letter reported recently that FDA personnel spent two weeks at Duke performing an onsite audit. The audit appears to focus on the absence of a necessary regulatory document called an Investigational Device Exemption (IDE), which should have been in hand before the gene test was used to direct patient care.
Duke spokesman Doug Stokke declined to tell me this week whether the auditors were still on campus, saying “We wouldn't comment on any kind of ongoing audit,” and no further information from the agency is yet available.
However, the documents revealed thus far do suggest that the institutional review board (IRB) at Duke was more accommodating of the researchers than one might hope, given that their primary goal should be to protect patients.
It appears that the IRB, for whatever reason, allowed Dr. Potti and his senior collaborator, Joseph Nevins, PhD, then Director of Center for Applied Genomics & Technology with Duke’s Institute for Genome Science and Policy, to proceed without the regulatory clearance document.
One of the documents the IRB signed off on had “n/a” written in the space where the IDE number should have been, and the space was simply left blank on another IRB form, according to the Cancer Letter, which obtained copies of their documents.
Yet, an IDE is not optional and likely would be required for such a test, according to Robert Becker, MD, PhD, Chief Medical Officer at the FDA’s Office of In Vitro Diagnostic Device Evaluation and Safety, who spoke at the December 20th meeting of the Institute of Medicine committee, which is charged with reviewing ‘omics-based tests in general and what happened at Duke specifically.
Why would an IRB be so lax? It is impossible to know with the information that is currently available, but one has to wonder if the committee was influenced in some way by the amount of money Duke was in line to gain if the trials were successful. (Similar genetic tests cost between $3,000 and $4,000, according to presentations at the JP Morgan Healthcare Conference by Genomic Health’s CEO and at the Biotech Showcase by Agendia’s CEO, which are the makers of Oncotype DX and MammaPrint, respectively.)
“It is always a problem when the institution stands to gain, especially in a huge fashion, financially from the research it is doing,” said Jerome Kassirer, MD, former Editor-in-Chief of the New England Journal of Medicine and an expert on conflict of interest. “There has to be some way of protecting patients against harm when lots of money is at stake.”
Kassirer thinks one way to improve the independence of IRBs in the future would be to include independent-minded individuals on the committee who are not otherwise associated with the institution, such as judges, lawyers, or community physicians.
“There ought to be a way to identify and include people on IRBs that can make a judgment independent of the money,” he said. “And I think you have to have a substantial representation of people who can say ‘No, this is not a good thing’.”
In fact, Duke IRB rules require that at least one committee member comes from outside Duke. I’m not sure, though, that one member on an 11-member committee would be enough, unless that person had unusual confidence and standing.
Subtle pressure for a favorable vote from the IRB may have also come from the amount of laudatory attention Dr. Potti’s work had received in academia and from the general media. If, initially, it sounds a bit far fetched that an IRB might bend under that sort of pressure, it seems less so after one reviews comments made to the IOM committee by NIH biostatistician Lisa McShane, PhD, in December. During a nearly hour-long presentation to the committee, she described her team’s efforts to clarify what had actually been done at Duke and determine if the findings were valid.
After her presentation, she was asked by a committee member why the cooperative group (CALGB) statisticians who reviewed a trial protocol based on Dr. Potti’s work didn’t notice the problems that she and her NCI colleagues did.
“Our experience was that the CALGB statisticians were basically in-tune to this and supported our position,” Dr. McShane responded. “But you have to appreciate that, as a statistician, you get involved with these research projects and you can be intimidated by the investigators who have very strong agendas. So [the cooperative group statisticians] didn’t want to be the ones coming out and saying ‘I think you need to do a validation study before you do this trial.’ NCI had to be the bad guy and take all the heat, and say ‘We’re not satisfied’.”
And she says, this isn’t the only time she’s encountered this type of pressure. “I’ve had numerous occasions where statisticians from the [cooperative] groups and elsewhere have said to me, ‘I’m glad you asking these questions. I tried raising these issues but I wasn’t listened to’,” Dr. McShane said.
It is unclear, as yet, if the IOM committee will address the issue of institutional conflict of interest, but I think we can be optimistic that they will based on a couple of comments and actions taken during their meeting in December.
Most important, one of the subcommittees formed will consider the roles and responsibility of the “actors” involved, and the institution was included in that list along with authors, funding agencies, and journals.
Also, several committee members appeared to think these issues were critical to fulfilling the committee’s mission.
“I think [the bigger issues] need to be [part of our effort],” said Gail Geller, ScD, Associate Professor of Pediatrics and the Bioethics Institute at Johns Hopkins Medicine in Baltimore.
“If we limit [ourselves] to the sort of data collection and analysis components of this whole story and ignore the larger principles related to research ethics and integrity, I think we will not be doing a complete root-cause analysis of what happened at Duke.”
Similarly, Tom Fleming, PhD, Ph.D., Professor of Biostatistics and Statistics at the University of Washington in Seattle, pointed to a recent paper of his in which he addressed issues of bias in medical literature. “There is enormous bias pervasive throughout all of health care for positive results, for being able to be viewed as making an advance,” he told the committee.
“And that has a huge implication, huge negative implications on the integrity of the research….Until we understand this and its pervasiveness and alter our scientific process accordingly, we are at continued great risk for biased conclusions.”
For his part, Dr. Kassirer, who is not involved with the IOM committee, emphasizes that institutional conflict of interest is not new. “It has been there from the beginning,” he said in a recent phone interview. He addressed it in a 2001 Boston Globe op-ed piece and in his 2005 book On the Take: How Medicine's Complicity with Big Business Can Endanger Your Health.
“It is an issue that has never been adequately addressed,” he continued, noting that there are two major reasons that it hasn’t been dealt with previously. “It only rears its ugly head rarely, as it did at Penn [in the Jesse Gelsinger case] and as it has done here. That is one reason.
"The second reason is that it is pretty well hidden. Nobody knows what the institutions are expected to profit by when some of these clinical trials go on.”
An IOM committee, though, has the expertise to tackle the issue, according to Dr. Kassirer and others. So I guess we should all cross our fingers and hope they do.