Integrating biomarkers into clinical trials: methodological issues for a new paradigm in nonsmall cell lung cancer

Abstract

Purpose of review 

To summarize the major methodological issues concerning prognostic biomarkers in nonsmall cell lung cancers (NSCLCs) and to discuss integration of biomarkers into clinical trials.

Recent findings 

Large phase 3 trials have recently been published in early-resected NSCLC with studies of biomarkers identifying subsets of patients that benefited most from the experimental perioperative strategy. The IALT-bio study reported that ERCC1 DNA-repair protein had prognostic and predictive implications. Other studies reported on the prognostic role of TUBB3 expression in stages I–III NSCLCs. The IPASS study reported the predictive and prognostic impact of epidermal growth factor receptor (EGFR) mutations in stage IV patients treated with EGFR tyrosine kinase inhibitor or platinum-based chemotherapy. Whereas EGFR mutations studies received prospective validation with trials in which treatment allocation was based on biomarker status, chemotherapy biomarkers still need such prospective confirmations, and many biases were still encountered in recent published studies.

Summary 

Biomarkers of treatment efficacy will help to tailor treatment in NSCLCs. EGFR mutations have reached that point. Markers of chemotherapy efficacy still need validation studies before coming into routine practice. Stringent methodological recommendations should avoid most of the possible pitfalls of those studies and allow clinical applicability of such biomarkers.

Introduction

Despite recent advances, lung cancer remains the main cause of cancer deaths worldwide ^[1]. Stage IV nonsmall cell lung cancer (NSCLC) is still an incurable disease with a median survival that reached a plateau of 10 months, with available cisplatinum-based regimens. Some subsets of patients benefit best from some specific drugs, either classical cytotoxics, or targeted therapies. The challenge remains to identify those subsets in order to give them the most individualized therapy for the highest survival. Even after early diagnosis and resection with curative intent of stages I–III NSCLC patients, occurrence of extrathoracic metastasis gives less than 50% 5-year survival rate. Meta-analyses only found a 5-year absolute benefit survival of 5–6% for perioperative cisplatin-based chemotherapy ^[2–4]. Therefore, in most stages I–II patients, perioperative chemotherapy is, at best, of no need in terms of survival, and sometimes detrimental in terms of quality of life or toxicity. In the IALT adjuvant trial, the 4.1% survival advantage observed in the chemotherapy arm means that 25 patients had to be treated for just one to benefit ^[5]. There is an urgent need to identify subgroups of patients that we could avoid exposing to a potentially toxic inefficacious treatment, or, on the contrary, patients that could actually benefit from adjuvant treatment. Biomarker studies have been therefore designed to identify which subsets of patients would be the most susceptible to take advantage from specific therapeutic strategies or drugs ^[6•].

Retrospective case series for training biomarker studies

Candidate biomarkers came from retrospective series of patients but most of those studies included less than 300 NSCLC patients, with multiple analyses, and could be only powered to detect a biomarker with high clinically significant influence ^[7••]. For many markers explored in retrospective cohorts, varying technologies have been used, with an obvious lack of standardization, with different positive value cutoff, leading to conflicting results, because key study characteristics differ across those studies. Populations in those retrospective studies could also be different taking account of confounding clinical or demographic variables. Therefore, those studies need to be considered as only hypothesis-generating and have to be validated by prospective studies, with an a priori statistical plan, in which the power should be ideally evaluated for each marker studied ^[7••]. This a priori power evaluation can only be performed if all technical issues of the assay measuring the marker are resolved, if the assay gives reproducible results across different retrospective case series, by different laboratory teams, leading to a precise estimation of the positivity rate, in a given well defined patient population. From published studies, only epidermal growth factor receptor (EGFR) mutations studies fulfilled at least a part of such strict methodological criteria, with still large variations in techniques used, but a reproducible frequency of 15% of EGFR mutations in white case series, with the use of sensitive allele-specific PCR-based assays and up to 40% in nonsmoking Asian patients ^[8,9••].

Prospective clinical trials for validating biomarker studies

The first clinical prospective randomized trials in NSCLC patients with ancillary biomarker studies were published within the last decade. Sometimes, even if the tumor banking was prospectively organized, the assay performance was not assessed in the context of a prospective trial, and reproducibility, sensitivity and specificity greatly vary from what was obtained with the same assay in small retrospective specimen sets, in which the biomarker hypothesis was initially raised. Stringent recommendations for standardization of prognostic biomarker studies reporting have been therefore recently proposed ^[10••]. However, most recent studies could simply not fulfill such stringent criteria, and still greatly methodologically differ from each other, obscuring clinical value, and currently impairing routine use of biomarkers in NSCLCs ^[11•]. The REMARK guidelines will help us to discuss some of the main methodological questions raised by prognostic biomarkers in NSCLCs.

Is the sample representative of the trial population?

Apart from rare trials that included the ability to perform the biomarker analysis as an inclusion criterion, the retrospective specimen collection rate has often been low in NSCLC prospective trials, with the risk that the biomarker study population differed from the whole population. In such studies, it actually remains questionable whether prognostic issues in a subset could be extended to the whole study population (selection bias).

If a majority of studies gives the patient flow diagram, only a minority provides a statistical comparison of both subsets, with or without biomarker analysis. In the IALT seminal adjuvant chemotherapy phase 3 trial, ERCC1 tumor staining was performed in only 41% of the whole population trial, from 28 centers that recruited more than 10 patients ^[12••]. Those centers contributed 867 pathological blocks out of 1045 included patients (83%) in those 28 centers, and in this subset, patients with or without ERCC1 analysis had similar clinical characteristics. However, no information was provided about possible clinical differences between those 867 ‘ERCC1 subset’ and the 994 remaining patients of the trial. Other trial collection rates were comparable or lower, ranging from 38% in the ALPI p53 immunostaining study ^[13] to 55% in the JBR-10 TUBB3 immunohistochemistry study ^[14], both of them giving disappointing results in their biomarker ancillary study. However, the p53 immunohistochemistry JBR10 study with a similar rate of analyses gave a positive result in terms of prognostic and predictive value ^[15]. The only neo-adjuvant chemotherapy phase 3 trial in early lung cancer, IFCT 0002, reported molecular analyses in 39% of the included patients for whom snap-frozen tumor preservation was possible, but TUBB3 IHC was possible in 78% of patients ^[16].

In recent studies of metastatic NSCLCs, the rate of analyzed specimen is even lower, because only small core biopsies are available for biological analyses and had to serve for both routine diagnosis and molecular studies. In the recent phase 3 First Line ErbituX (FLEX) in lung cancer trial ^[17•], only 35% from 1129 patients could have a K-Ras mutation analysis (although K-Ras was supposed to be a resistance marker to cetuximab, an anti-EGFR monoclonal antibody), and only 25% had EGFR FISH analysis. At no point was any comparison between the ‘molecular subset’ and the whole study population provided. Indeed, the sponsor only collected slides for EGFR immunostaining because positive EGFR expression was an inclusion criterion, but this small material was obviously unsuitable for the molecular techniques used. Pathological blocks collection rate was also low in the IRESSA NSCLC Trial Evaluating Response and Survival versus Taxotere (INTEREST) ^[18], the IRESSA Pan-Asia Study (IPASS) ^[19••] and the Sequential Tarceva in Unresectable NSCLC (SATURN) ^[20] trials with oral EGFR tyrosine kinase inhibitors (TKIs), ranging, for EGFR mutation analyses, from 36% (IPASS) and 41% (SATURN) to 20% (INTEREST), still representing 297, 367, and 437 EGFR molecular analyses, respectively. In those trials, patients with EGFR mutations and treated with gefitinib or erlotinib had significantly better PFS than patients receiving chemotherapy (IPASS and INTEREST) or placebo (SATURN) with low hazard ratios for progression, ranging from 0.48 (IPASS) to 0.16 (INTEREST) and 0.10 (SATURN). Thus, despite the low rate of specimen collection the high predictive value of EGFR mutations, with hazard ratios under 0.50 for event, being unusual in NSCLC, those biomarker studies were positive at least for PFS prediction. Another historical example is the molecular posthoc analysis of the phase 3 BR21 trial that randomized placebo versus erlotinib in second-line setting for metastatic NSCLC ^[21]. Only 24% of the patients (177/731) recruited could have EGFR mutation analysis, which failed to predict better prognosis in mutated patients treated by erlotinib ^[21]. However, it should be remembered that this subset significantly differed from the whole study population by racial repartition (more Asian patients), by number of prior treatments (reflecting a more indolent tumor evolution in this subset), by the best response observed to previous treatments (with more responder patients), and by the time from diagnosis to randomization (with more patients recently diagnosed).

Is the sample representative of the disease treated in this trial?

In prognostic molecular studies, the specimen chosen for analysis could induce false interpretation simply because it only reflects a small part of the tumor which is often oligoclonal. Thus, a core biopsy could differ from the whole tumor, or from distant metastases. Alternatively, the group of patients submitted to molecular analysis could be falsely presented as representative of a particular clinical group, leading to an ‘over-interpretation bias’ if molecular features are thought to be prototypic of a different population.

Biopsy versus surgical sample (metastatic versus early-stage nonsmall cell lung cancer studies)

In metastatic NSCLC, one of the main issues is the type of specimen used for molecular studies. In these patients, histological diagnosis mainly relies on core biopsies often containing 100 tumor cells, and sometimes representing less than 20% of the tissue biopsy. Taking into account tumor heterogeneity, and oligoclonal nature or lung cancer proliferation, it is questionable whether those core biopsies are actually representative of the whole primitive tumor, or of the different metastatic clones that could arise in brain, bone or liver. Some rare studies addressed that question, systematically comparing molecular analysis of core biopsies and surgical specimens in the same patients, and some histochemical markers were shown to considerably vary across the different types of pathological specimens analyzed ^[22•]. A lower number of studies compared primitive NSCLC tumors and metastases, occurring either synchronously or later in the disease evolution. In such studies, driver-mutations, such as EGFR and p53 mutations detected in core biopsies, or in the primitive surgically removed tumor, are generally also detected in corresponding surgical specimens, in distant metastases or in synchronous clonal tumors, respectively ^[23–26], whereas the same features are seen for K-Ras mutations but with data coming mainly from colorectal cancer ^[27]. Conversely, gain of such driver-mutations appears to be rare in metastases, or after treatment from initially mutation-negative primitive tumors. However, gain of additional nondriver molecular events seems possible in distant metastases, such as gain or loss of chromosomal material, amplification or deletion of critical metastasis genes.

Progressive or responder patients (BR21 versus ISEL)

After the BR21 seminal phase 3 trial that led to erlotinib registration for second-line therapy of metastatic NSCLC patients, it was claimed that erlotinib could be indicated in progressive patients with cisplatin-based chemotherapy ^[28]. However, in BR21 a large subset of patients were in fact responder or nonprogressive patients after four courses of chemotherapy, who were immediately recruited after four initial cycles of chemotherapy, and therefore potentially had chemotherapy-sensitive tumors, indicative of a specific gene expression pattern. Conversely, in the similar trial ISEL that randomized gefitinib versus placebo in second-line setting, a large majority of patients were progressive patients, and therefore have probably more aggressive chemotherapy-refractory tumors ^[29]. Therefore, biomarker analysis findings in BR21, claiming that EGFR wild-type patients, patients with high polysomy or EGFR gene amplification, or patients with positive EGFR immunostaining, benefited from erlotinib, as the whole study population, should not be extended to progressing patients, in whom this trial was not designed to show any advantage of EGFR TKI. The SATURN trial in which only nonprogressive patients were recruited after cisplatin-based treatment, subsequently reproduced the same results as BR21 in its ancillary biomarker study, with the notable exception of a predictive impact of EGFR mutation, possibly explained by a more robust assay for EGFR mutations detection and the noninclusion of progressing patients ^[20].

Is the method robust, reproducible, widely accessible, cost-effective?

In NSCLC patients, molecular diagnosis often relies on small bronchial biopsies only containing 100 tumor cells surrounded by stromal tissue. Molecular assays need therefore to be sensitive, discriminating tumor-mutated cells from surrounding normal tissue, but also easy to perform in every lab, without technically sophisticated expensive devices. Two historical examples can illustrate the need to use validated methods adapted for clinical routine use.

Epidermal growth factor receptor mutations assay in nonsmall cell lung cancer

The BR21 phase 3 trial randomizing erlotinib versus placebo in second-line setting for stage IV NSCLC reported surprisingly that EGFR mutations had no impact on survival in patients treated by erlotinib ^[21]. A close look at the mutational events initially reported in the BR21 trial showed a much higher frequency of rare EGFR mutations that were not reported in previous or subsequent EGFR molecular studies. It was experimentally shown later that deamination of cytosine or adenine DNA residues, induced by fixation and paraffin-embedding, led to a classical experiment artifact in low-concentrated DNA, accounting for such a biased EGFR mutations representation ^[30••]. Clearly, the investigators should have validated their assay in a retrospective case series of small-sized paraffin-embedded core biopsies, to fix such an artifact, by adapting DNA template concentrations to their PCR-based sequencing reaction. Sensitive and specific allele-specific oligonucleotide PCR or DHPLC assays were subsequently developed, that only amplify mutated nonaltered tumor DNA, and avoid such artifacts in low-concentrated tumor DNA ^[31].

Promoter gene methylation assays

Methylation studies of promoter genes such as tumor-suppressor gene RASSF1A have also been questioned. In the IFCT 0002 phase 3 trial ^[32], a robust, cost-effective methylation specific-PCR assay previously published was used, giving reproducible results in two independent labs in the same series of patients, and showing a prognostic and predictive role for RASSF1A ^[33]. More quantitative techniques have been recently used such as pyrosequencing, that gives a precise additional information about the promoter region length being methylated, that is the promoter gene methylation intensity. For promoter gene methylation assays, the main issue remains how methylation correlates with gene expression. In Bio-ICT0002, MS-PCR was shown to correlate with statistically significant downexpression of RASSF1A protein as detected by western blotting analysis of tumoral protein extracts ^[33]. Of the six major CpG islands promoter, methylation level percentage is less than 20% and is enough to affect RASSF1A protein expression ^[34•]. Conversely, methylation level at the MTHFR promoter gene needs to be higher than 80% to downregulate MTHFR mRNA. For such a gene, there would not be a strict correlation between moderate levels of methylation as detected by sensitive MS-PCR assays, and protein expression. Therefore, methylation assays need to be correlated to gene expression for each specific gene. For some genes, a simple and robust MS-PCR assay would be enough, for other genes a complete promoter gene tedious pyrosequencing could be necessary.

Is the method statistically sound and the results strictly confirmed by an internal or external validation method?

Once the question of variability of molecular assays is fixed across different labs (an issue which is, of course, not trivial), limiting the confusion bias is a major step of the prognostic analysis, by a multivariate analysis controlling for stratification factors of the trial, and for the confounding clinical or pathological characteristics influencing survival in the univariate analysis, with a given P value (inferior to 0.1 or 0.2, depending on the studies).

Then, the next point is to check that the results obtained in one specific series of patients simply could not have been obtained by chance (sampling bias), taking account of possible variations across different series of patients, from various geographical origin, smokers or nonsmokers, chemotherapy-naive or not, with varying sex ratios or ages (reproducibility of the biomarker prognostic influence).

False-positivity results risk limitation

The main methodological artifact is the multiplicity of analyses performed within a single series of patients. IALT-bio is, to our best knowledge, the only trial listed in this review that mentioned an a priori power calculation for biomarker study. The authors specified the need to analyze at least 800 specimens to provide a 66% power to detect a 20% survival difference at 5 years (hazard ratio = 1.2), with a two-sided 1% alpha level, taking account of 11 clinical and prognostic factors and three factors used in the stratified randomization. IALT-bio investigators reported indeed multiple immunohistochemical analyses (at least 10) comprising ERCC1 nucleotide excision repair protein, p27, cyclin D1, cyclin D3, cyclin E, Ki-67 cycle proteins, TUBB3 tubulin protein, MRP1 and MRP2 multidrug-resistance proteins, and finally MSH2 DNA mismatch repair protein, leading to a global model with not less than 24 different variables ^{[12••,35–37]}. Additional variables should be added, because grouped analyses ERCC1/MSH2 and MSH2/p27 were also reported. All reported P values considered as significant were indeed 0.01 or less, but one (low MSH2 prediction of a longer survival, P = 0.03). However, two sequential analyses were performed at 5 and 7.5 years of median follow-up. In this latter analysis, low ERCC1 content was still associated with better survival in patients who had received adjuvant chemotherapy, but with a substantial increase of hazard ratios between the 2006 and 2008 analyses, whereas the impact of adjuvant chemotherapy on OS was lost at 7.5 years of median follow-up, in the whole clinical study population ^[38].

Some methodological techniques have been proposed to correct the potential effect of multiple analyses, and limit the risk of false-positive results, such as Bonferroni Holm's or Hochberg's correction, widely used for cDNA micro-array analyses ^[39]. Such methods are taking account not only of the multiplicity of variables, but also of the number of sequential analyses performed during the period of follow-up. However, to our best knowledge, only the Bio-IFCT 0002 study reported the use of such a statistical correction, in a prospective biomarker study in NSCLC, re-enforcing the prognostic and predictive value of RASSF1A methylation.

External and internal validation

Reproducibility of prognostic biomarker studies is a major issue, because the ultimate goal of developing a predictive biomarker is indeed to use this biomarker in a much larger, broader and possibly heterogeneous population than the initial set of patients in which it was first studied. One possibility is to reproduce the molecular assay analysis in an independent series of patients, from a distinct research group, but apparently comparable to the initial series, in terms of all identified clinical and pathological variables. This ‘external’ validation has mainly been used in micro-array studies of NSCLC specimens ^[40], because a complete lack of concordance across studies was previously reported ^[41].

When comparable external series of patients are lacking, some methods have been proposed giving an ‘internal’ validation. The simplest is to randomly select within the whole series of patients a training set from which a multivariable prediction model including the biomarker(s) of interest and the clinical predictors is derived. Then, this model is tested without any change in the rest of the same patients series, used as a validation set. Such a strategy needs a large initial sample size, to get precise estimates. Other more complex methods have been proposed, such as ‘leave-one-out’ crossvalidation approach, or the bootstrap method ^[42,43]. Bootstrapping consists of drawing samples (with replacement) from the original data set to generate a large number of training sets (several hundred) of the same size as the original sample. A prediction model is developed on each training set and tested on the original data. Validation results are then reported as the average performance over the whole process.

Unfortunately, recent biomarker studies in NSCLC did not include such validation procedures in phase 3 clinical trials.

Conclusion

This review only focused on several methodological issues of prognostic and predictive studies of biomarkers in NSCLC clinical trials. Biomarker analyses in retrospective series of patients are still needed to set up a biological robust, reproducible assay, and to generate hypotheses that need to be prospectively validated in broader series of patients provided by large clinical trials. Such validations need a large specimen collection, to assure a minimal sample size for a minimal statistical power to detect a significant and clinically relevant difference in survival, according to the biomarker status. Multivariate prognostic analysis of several biomarkers including all clinical, pathological prognostic variables and stratification factors of the clinical trial needs statistical corrections for multiple analyses to avoid false-positive result reporting. At best, external validation in comparable series of patients is suitable but rarely possible. Alternatively, internal crossvalidation complex procedures could assure the stability of the biomarker. If all those recommendations have been edited by the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics, their systematic use remains rare in NSCLC, still precluding the clinical routine applicability of prognostic studies in clinical trials.