DENVER—Projects aimed at developing biomarkers to diagnose cancers and predict their response to targeted drugs are threatened by a most fundamental aspect—unreliable biospecimens. So said Carolyn Compton, MD, PhD, Director of the National Cancer Institute's Office of Biorepositories and Biospecimen Research, speaking here at the AACR International Conference on Molecular Diagnostics in Cancer Therapeutic Development.
“It has been widely agreed that the lack of high-quality, clinically annotated human specimens is the number one limiting factor for translational research,” she explained.
Dr. Compton told the meeting, which attracted leaders and scientists in academia and the biotech industry, that the quality of data from translational research can never be higher than the quality of the biospecimens going into the front end—“And I'm here to say that, sadly, we do not have what we need to do this efficiently.”
Dr. Compton's statement was echoed in a recent AACR-FDA-NCI consensus report on biomarkers, which ranked improvement in the quality and consistency of biospecimens as a top priority for action (OT, 10/10/10).
In her presentation, titled “The Right Stuff: The Need for Standardized Human Benchmark Samples as the Yardstick of Truth for Assay Development,” Dr. Compton said the paucity of high-quality biospecimens and the use of low-quality analytes was precisely the reason her office was created at the NCI.
Case in Point: Cancer Genome Atlas Project
She used the Cancer Genome Atlas project as a case in point. In its pilot phase the project focused on glioblastoma multiforme, serous carcinoma of the ovary, and squamous cell carcinoma of the lung. Specimens from biorepositories went to 10 participating institutions using different analysis platforms to research the exact same biomolecules extracted from exactly the same samples, from exactly the same tumors and from exactly the same patients.
“The rate of failed specimen quality was so enormous that we only reached our accrual goals for ovarian cancer, we never got to our accrual goal for glioblastoma multiforme, and we didn't even start the squamous cell carcinoma of the lung,” Dr. Compton said. “Even among the samples that passed pathology quality control, 30% failed molecular quality control in central laboratory.”
She said the Cancer Genome Atlas project showed that the quality of existing samples in biobanks across the country and around the world is wildly over-estimated; that the collection of normal samples is not routine; that clinical data on specimens are not readily available; and that histologic quality does not guarantee molecular quality.
You can have the perfect assay and still get the wrong answer if you mess up the biospecimen, she said. Human biospecimen handling pre- and post-acquisition can alter its molecular profile, its molecular quality, and can produce the wrong immunohistochemical results and skewed clinical chemistry.
For the patient, there is a dangerous potential for incorrect treatment if that treatment is based on flawed biomarkers.
“HER2 is a powerful example of this,” Dr. Compton said. “The decision to give Herceptin to a patient is entirely dependent on the result of the assay on the biospecimen without any other clinical information or input from any other source.
“So therapy is directly linked to a diagnostic test, and if you get either a false-positive or a false-negative [on HER2 status]—which we learned happened until recently about 20% of the time—you are subjecting a patient to a very expensive therapy that may not have any effect on their tumor.”
Alternately, an incorrect diagnostic test on a biospecimen may result in withholding a potentially lifesaving therapy from a patient who merits it, she said.
“The effects on outcomes in the research world—the Journal of Irreproducible Results—are rife with these, while the misinterpretation of artifacts as biomarkers is already rampant in our field,” she said.
Researchers can turn this around and improve the quality of biospecimens, she said, but they first must understand the biospecimen.
“It's a living object, it has a life of its own,” Dr. Compton said, noting that this is unlike the view of most investigators that the biospecimen simply represents a little “mini-me” of the patient which is given to the translational researcher, and industry turns this into a product that's given to the MD and the patient is cured.
“In fact, the biospecimen is subject to molecular and biologic stresses that are enormous,” she said, starting with preoperative anticancer treatment and continuing with how the biospecimen is handled when removed from the patient.
“We need to study biospecimens as their own objects of research.” Dr. Compton said her office is now funding a first-of-its-kind scientific program where the human biospecimen is actually the focus of the research.
For example, researchers have learned that the longer the time from collection to biospecimen fixation, the greater the diminution of the signal.
“The test that we thought was the most stable, in situ hybridization, is greatly affected by time to fixation,” she said. In some cases the diminution of the signal on the diagnostic test is greater with time, but in other tests, such as phosphoprotein kinase immunohistochemistry in colon cancer, the signal increases in expression with time to fixation.
Postsurgical ischemia and gene expression can also dramatically affect different classes of genes, she said.
In the world of molecular biomarkers, using fluid specimens from plasma—the raw materials on which the whole field of proteomics rests—it is known that there are enormous numbers of procedural variables, she said. Venapuncture, phlebotomy, collection devices, processing devices, time between collection and storage and storage and shipping conditions themselves all dramatically change the patterns of proteins that are detected in a proteomics test.
Vote of No Confidence
Dr. Compton said her office was “aghast” at the findings of a poll of 7,000 investigators funded by the National Cancer Institute who consider themselves doing translational research, when the overwhelming majority said the quality of samples they needed was “somewhat difficult,” “difficult,” or “very difficult” to obtain.
“And in an even sadder set of data, they told us that they doubted the quality of their own data because they doubt the quality of the specimens they are using,” she said. “They said most of the time they use any specimen they can get—which we now call “convenience samples”—and simply design their experiments around any specimens they can get ahold of, regardless of quality.”
“We are filling our pipelines with questionable samples, and our researchers know it.”
Considerations to Ponder
Dr. Compton offered some considerations to ponder from the FDA's point of view:
- Human biospecimens are unregulated, even though the tests performed on them are regulated. “We worry more about the grades of beef than the quality of the samples, because the FDA regulates the quality of beef,” she said.
- Research biobanks are unregulated. “The NCI has put out best practices to tell people how to improve their collection processing and storage of samples in a standardized way, but the NCI is not a regulatory agency, we cannot make anybody do anything,” she said. “But the FDA can, and in fact the FDA may start to regulate biobanks if we as a community cannot do this successfully on our own.
- There is no clinical requirement at all, through any clinical regulatory agency approvals or accreditation body, to control or report biospecimen handling “lifecycle variables” in clinical laboratories. “Formalin-fixed passive-embedded tissues—the fuel that drives most of the science in our clinical research enterprise—is mostly mistreated because it comes from hospitals across the country and is processed only according to the clinical standards that exist in that hospital which are different from hospital to hospital and which are never regulated by any authoritative body.”
Dr. Carolyn Compton noted that the lack of high-quality, clinically annotated human specimens actually served as a “call to action” that brought her to the National Cancer Institute from academic medicine. The problem, she noted, is spelled out very well and in great detail in a recent article in Wired, “Libraries of Flesh: The Sorry State of Human Tissue Storage” by Steve Silberman: http://www.wired.com/magazine/2010/05/ff_biobanks