Ipilimumab was the first immune checkpoint inhibitor that showed efficacy in treating metastatic or unresectable melanoma . A phase III study of previously treated patients showed a hazard ratio (HR) for overall survival (OS) of 0.66 [95% confidence interval (CI): 0.51–0.87] for patients treated with ipilimumab alone compared with the control of a glycoprotein 100 (gp100) peptide vaccine, and with similar results for patients treated with both ipilimumab and gp100 compared with the control (HR: 0.68, 95% CI: 0.55–0.85) . Analyses of subgroups suggested limited heterogeneity of treatment effects. Subsequent results from the CA184-024 first-line study comparing ipilimumab + dacarbazine with placebo+dacarbazine reported similar OS benefit (HR: 0.72, 95% CI: 0.59–0.87) . A pooled analysis of ipilimumab trials either in the first-line or the previously treated metastatic or unresectable setting revealed 3-year survival rates between 20 and 26% with a plateau of the survival curve at that point .
As median survival for metastatic melanoma patients treated with chemotherapy alone is very poor (6–9 months), ipilimumab was a major breakthrough . Current population-based studies suggest that the new generation of targeted and immune-based therapies has improved survival times for metastatic melanoma patients since 2010 [6,7]. However, these studies compared results by treatment era, where multiple types of systemic therapy (e.g. chemotherapy, anti-CTLA4, BRAF/MEK, and anti-PD-1) were used among patients with variation according to the era. Comparing outcomes by era is an ecological approach, and does not provide a direct patient-level comparison of specific regimens. There is insufficient population-based evidence comparing specific regimens such as ipilimumab with specific chemotherapy controls (e.g. dacarbazine). Population-level evidence is important, as results of randomized controlled trials may not generalize to the population at large. Trial results may be superior because of restricted entry criteria (e.g. no brain metastasis and lower comorbidity) .
We sought to estimate the effect of first-line ipilimumab in the province of Ontario using administrative data. Administrative data refer to information primarily collected as part of the operation of programs by governments and other organizations. Given the many variables that could be used for adjustment in the available provincial administrative data, we set out to explore a novel double-selection method, a machine learning technique, to estimate treatment effects while controlling for the important confounding variables. Time-dependent treatment usage was also investigated, given the use of multiple lines of systemic therapy in contemporary practice for metastatic melanoma.
Patients and methods
This was a population-based study of patients treated for metastatic or unresectable melanoma with first-line ipilimumab or dacarbazine between 2007 and 2015 in Ontario. Ontario is the largest province in Canada with a population of 13.6 million people. Melanoma patients were identified in the Ontario Cancer Registry . Patients were identified on the basis of documented treatments from the cohort of patients receiving palliative treatment for melanoma (systemic therapy, radiotherapy, and/or metastasis surgery) in Ontario (2007–2015) . Cohort characteristics have been previously described . Treatments were classified through administrative data housed at the Institute for Clinical Evaluative Sciences. The Ontario Cancer Registry data capture incident cancer cases in the province of Ontario with a high level of completeness overall [9,10]. Patients with nonmelanoma cancer diagnoses were excluded.
The control group was those treated with first-line dacarbazine (2007–2009). The ipilimumab cohort was those treated with first-line ipilimumab (2010–2015). Included in the ipilimumab cohort were patients who received chemotherapy for less than 60 days and switched over to ipilimumab. This occurred during a brief time in Ontario when exposure to chemotherapy was required before funding for ipilimumab. These patients were considered to essentially be recipients of first-line ipilimumab treatments, as there was only transient treatment with chemotherapy ( < 60 days). There were no other exclusion or inclusion criteria for the study cohort.
Classification of independent variables
Intravenous systemic therapy agents, radiotherapy, and cancer surgery are provided in Ontario through a single-payer, publicly funded cancer system coordinated by Cancer Care Ontario.
A comprehensive set of patient-linked data was available in the administrative data set used for this study. This included systemic drug regimens, surgical and radiotherapy procedures, as well as demographic and geographical information. For consistency, the treatment information was censored on 31 December 2015, beyond which data completeness was variable. Provincial reimbursement data from specialist physicians administering systemic therapy was utilized up to 31 August 2016, to identify cases treated with systemic therapy without a named drug regimen, in particular, treatments occurring beyond the treatment information censoring date.
Patient characteristics at the time of first systemic treatment and disease characteristics of the original melanoma diagnosis were described. Demographic data were obtained from the Ministry of Health and Long-Term Care administrative data and linked to other sources. Socioeconomic status was based on neighborhood household income quintiles. Rurality of patient residence at the time of first systemic treatment of metastatic or unresectable melanoma was characterized by the 2008 Rurality Index for Ontario . The Elixhauser comorbidity score was utilized with a 5-year look back with the Canadian Institute for Health Information Discharge Abstract Database and Same Day Surgery Database . Diagnosis codes for cancer metastasis or solid tumor without metastasis were not counted in the score.
To capture a large range of possible confounders of the relationship between treatment and survival times, all continuous variables were transformed so that they included a square, a square-root, and a discretization into five quintiles (i.e. a dummy variable equal to one if it fell into a given quintile). This allowed for the creation of a total of 166 dummy variables; 73 were sociodemographics (e.g. age, sex, Elixhauser comorbidity, and place of residence), 79 disease variables (e.g. stage, topography, morphology, place of diagnosis, and time from first palliative treatment), and 14 treatment variables (relating to chemotherapy, radiotherapy, and surgery). A simple screening procedure to remove highly unbalanced and correlated features (i.e. correlation >0.90) was then implemented. After these steps, 86 dummy variables remained.
During the time of rapid change in systemic therapy of the study period, many patients received multiple lines of treatment, and the sequencing of use of the more effective, newer agents varied over time. For example, anti-PD-1 agents may have been used after ipilimumab, or ipilimumab may have been used after the failure of multiple lines of chemotherapy. For this reason, time-dependency of treatment was considered in the survival analysis when evaluating the effect of ipilimumab versus dacarbazine. The time-dependent model took into account not only the use of other treatments apart from the primary exposure (e.g. second-line BRAF inhibitors, anti-PD-1 treatments, radiation, or metastasis surgery) but also the impact of the time when these treatments were administered, in relation to other treatments.
Classification of dependent variables
OS time was the primary endpoint. It was measured from the time of first palliative systemic treatment to death. Maximum follow-up was until 31 March 2016. Patients still alive at that time were censored.
The treatment effect of ipilimumab was estimated in terms of a HR by the semiparametric Cox proportional hazards model. To control sufficiently for important confounding effects while excluding extraneous variables that would compromise the efficiency of the model, a modified Lasso algorithm was employed in the double-selection method [13,14]. This technique selected important variables related to both treatment assignment and survival times in the Cox proportional hazards model. The selection is governed by a penalty that adjusts the trade-off between possible inclusion of unnecessary variables, and exclusion of true confounding variables under the assumption that these two sets of variables are uncorrelated with each other on average. The value of the penalty is set, so that the average number of selected variables having no true confounding effects on the relationship between the treatment and survival was predicted to be less than or equal to five for the primary analysis. Adjusted subgroup analyses were undertaken using the same model selection procedure. However, given the smaller sample sizes of the subgroups, there was a greater risk of over-parameterizing the model, and so a more stringent penalty was utilized, with an estimated average number of selected variables having no confounding effect expected to be no more than one. There were insufficient patient numbers with autoimmune conditions or brain metastasis treatment before the first-line systemic therapy to perform subgroup analyses on these groups.
For sensitivity analysis, the double-selection approach was compared with a traditional forward selection approach, using a 1% P value threshold as well as a simple univariate analysis. The P value threshold was chosen to ensure that the forward selection approach selected a reasonable number of covariates to make its variable selection comparable to the double-selection procedure’s expected false-positive rate.
Of the 2793 patients receiving treatment for metastatic or unresectable melanoma between 2007 and 2015 in Ontario, 464 patients were identified who received first-line dacarbazine or first-line ipilimumab. There were 175 patients who received first-line dacarbazine. Among them, 41 received dacarbazine combined with a platinum agent, and/or other cytotoxic agents. There were 289 patients who received first-line ipilimumab. Of them, 117 received ipilimumab within 60 days of the first-line chemotherapy. Reflecting the timeline of known trials and provincial approval dates, ipilimumab ( < 60 days chemotherapy) was administered primarily during the period spanning from 2012 to 2014, and first-line ipilimumab with no initial chemotherapy in 2015. Patient and health system characteristics were similar between the cohorts (Table 1). Details of the first-line ipilimumab patients subdivided into whether or not initial chemotherapy of less than 60 days was received are provided in Supplementary Appendices 1–2 (Supplemental digital content 1, http://links.lww.com/MR/A116). Overall, the baseline characteristics of these subgroups were similar. Reflecting the population distribution in Ontario, 59.3% of all patients lived in the Greater Toronto Area and Central Ontario. There was a predominance of patients from higher income quintile neighborhoods (e.g. 4th and 5th income quintiles). The median age was slightly higher in the ipilimumab cohort compared with the dacarbazine cohort (i.e. 64 vs. 62 years). Most patients had little or no identifiable comorbidity (85.4% Elixhauser score 0/1).
Although primary site distributions were relatively similar, there were less unspecified primary sites in the first-line ipilimumab group, which represented the most contemporary group of patients (Table 2). Overall, there were less than 10% of patients presenting with noncutaneous melanoma. The median time from the first (any stage) melanoma diagnosis to the first palliative systemic treatment varied: 591 days for first-line dacarbazine and 712 days for first-line ipilimumab. Notably, there were more patients who received palliative radiotherapy or metastasis surgery before the first-line ipilimumab. Subsequent lines of chemotherapy varied between groups. First-line dacarbazine was most often followed by other chemotherapy in subsequent lines. Use of new agents was uncommon, with anti-CTLA4 used in 8.6% (Table 3). In comparison, subsequent anti-PD-1 therapy was used in 17.3% of the first-line ipilimumab patients. However, 29.8% in the first-line ipilimumab group had systemic treatment billings more than 8 weeks after receipt of the last named regimen, suggesting many more may have had anti-PD-1 or other agents beyond the dates covered by the available drug databases (Table 3). This was more common among the first-line ipilimumab patients without previous chemotherapy, 73% of whom started ipilimumab in 2015 (Supplementary Appendix 3, Supplemental digital content 1, http://links.lww.com/MR/A116). The receipt of unnamed systemic agents after the last named regimen was included as a variable for the adjusted analysis. Use of radiotherapy after the first-line systemic therapy was notably less in the first-line ipilimumab group, although follow-up time from the first systemic treatment was shorter. For the first-line ipilimumab cohort, the use of previous and subsequent metastasis surgeries was more common than for the dacarbazine cohort (Tables 2 and 3). Emergency department visits and hospitalizations within 3 months of the first-line treatment start were similar between first-line dacarbazine and first-line ipilimumab (Table 3). However, when the first-line ipilimumab group was stratified by receipt of previous chemotherapy, the ipilimumab less than 60 days chemotherapy group had more patients with two or more hospitalizations (19.7 vs. 8.1%) or emergency department visits (18.8 vs. 7.0%) (Supplementary Appendix 3, Supplemental digital content 1, http://links.lww.com/MR/A116). Findings may reflect temporal changes in acute toxicity management or possibly unmeasured differences in case mix.
Overall survival analysis
Median follow-up was 5.2 months (range: 0.2–107.6 months) for first-line dacarbazine and 7.0 months (range: 0.3–43.9 months) for first-line ipilimumab. Median survival was 5.2 and 9.5 months, respectively. For first-line dacarbazine, 1-year OS was 18.9%, 2-year OS was 7.4%, and 3-year OS was 4.0%. For first-line ipilimumab, 1-, 2-, and 3-year OS were 46.5, 29.8, and 24.4% respectively. As illustrated in Fig. 1, survival was greater in the first-line ipilimumab subgroup with no previous chemotherapy, compared with those receiving less than 60 days of chemotherapy before ipilimumab. Stratified by the receipt of chemotherapy, 1-, 2-, and 3-year OS were 54.4, 49.2, and 45.1% for ipilimumab with no previous chemotherapy and 37.6, 17.8, and 13.0% for ipilimumab with less than 60 days chemotherapy, respectively.
The unadjusted treatment effect of first-line ipilimumab for OS was 0.52 (95% CI: 0.40–0.67; P < 0.001). The adjusted hazard ratio (AHR) comparing ipilimumab with dacarbazine was 0.63 (95% CI: 0.47–0.84; P = 0.002). Details of the multivariable model selected by the machine learning algorithm are provided in Supplementary Appendix 4 (Supplemental digital content 1, http://links.lww.com/MR/A116). Notably, differences in time from the first melanoma diagnosis to the first-line treatment (lead time bias) were controlled for in the model. Traditional forward stepwise selection produced similar results (AHR: 0.72, 95% CI: 0.55–0.95). Similar adjusted results comparing ipilimumab with dacarbazine were also obtained when the ipilimumab group was limited to ipilimumab patients with no previous chemotherapy (AHR: 0.63, 95% CI: 0.44–0.90; P = 0.012) or to ipilimumab with less than 60 days of previous chemotherapy (AHR: 0.68, 95% CI: 0.47–0.97; P = 0.034). Variables in the models are provided in Supplementary Appendix 5 (Supplemental digital content 1, http://links.lww.com/MR/A116).
Ipilimumab was associated with improved OS across all subgroups (Fig. 2). However, the estimated effect was greater in those with age younger than 65 years, male sex, without comorbidity, and those living in neighborhoods ranked in the lowest two income quintiles. There was an insufficient sample size to test appropriately for interactions between treatment effect and subgroup.
Discussion and conclusion
Summary of findings
In our population-based study, we found a substantial improvement in population-based OS with first-line ipilimumab compared with first-line dacarbazine. This was consistent with clinical trial results. The estimated treatment effect was robust to variation in covariates in two distinct ipilimumab-treated populations (first-line ipilimumab with no previous chemotherapy or ipilimumab with < 60 days of previous chemotherapy). There was a plateau in the survival curve after 2–3 years, similar to trial results. Furthermore, subgroup analysis suggested a treatment effect in favor of ipilimumab even among groups underrepresented or understudied in trials (e.g. older patients, females, patients with comorbidity, patients from lower income quintile neighborhoods, and patients based on previous interferon treatment).
This study provides an important contribution to the melanoma literature, as to our knowledge, it is the first covariate-adjusted population-based study of a proven immunotherapy drug for metastatic or unresectable melanoma compared with historic chemotherapy. Because of the multiple linked population-based administrative data sources, we were able to avoid selection biases that may be associated with institutional case series, extended access program reports, or reports on selected patients or subpopulations. It allowed us to include the full age and performance status spectrum of melanoma patients and patients with brain metastases. Our adjusted analyses took into account subsequent lines of therapy, including investigational agents, special access programs, and publicly funded agents, to reduce confounding biases when estimating treatment effects. Our population-based analysis showed similar mortality reductions to those reported in the CA184-024 trial of first-line ipilimumab with HR of 0.63 (95% CI: 0.47–0.84) and 0.72 (95% CI: 0.59–0.87), respectively .
Nonetheless, we note that the absolute survival for ipilimumab ( < 60 days chemotherapy) at specific time points was less than observed in CA184-024 . For example, 2-year survival was 17.8% with dacarbazine and placebo in the trial, compared with 7.4% with dacarbazine in our population-based sample. For ipilimumab + dacarbazine, 2-year survival was 28.9% in the trial, compared with 17.8% with ipilimumab ( < 60 days chemotherapy) in our sample. Variation in patient factors, disease factors, and treatment factors are probably involved. We note, for example, that our median age was 65 years for ipilimumab ( < 60 days chemotherapy) compared with 57.5 years in the ipilimumab arm of the randomized trial. We observed a difference in treatment effect for those younger than 65 years of age compared with those older than 65 years of age, although, in our small sample, this was not significant (Fig. 2). There were also over 7% of patients with ocular or mucosal melanoma in our sample; such patients were excluded in the trial. Similarly, patients with brain metastasis were included in our cohort but excluded in the trial. Differences in available subsequent lines of treatment and time from diagnosis to the first-line systemic treatment may have also influenced observed survival trends.
However, absolute survival for first-line ipilimumab (with no previous chemotherapy) was much greater than in CA184-024 (Fig. 1), with a 2-year survival of 49.2%. This was most likely influenced by subsequent availability of anti-PD-1 treatment for first-line ipilimumab patients without previous chemotherapy, to a greater degree than for the earlier cohort of ipilimumab ( < 60 days chemotherapy) patients. Despite the large differences in unadjusted survival between these cohorts, multivariable modeling techniques utilized in this study resulted in very similar estimated treatment effects of ipilimumab for both groups.
We also observed differences in ipilimumab treatment effect between sexes, with males having a lower risk of death than females (male AHR: 0.58, 95% CI: 0.40–0.85; female AHR: 0.78, 95% CI: 0.52–1.19). This is in keeping with randomized trial data, and the meta-analysis by Conforti and colleagues of immunotherapy treatment effect stratified by sex [2,3,16]. However, interpretation of our study findings is limited by sample power. Reasons for these differences are unclear. Future studies may lead to the improved selection of patients for immunotherapy.
Our paper contributes to the small but growing field of machine learning techniques that are being applied to statistical inference problems. Specifically, we modified the Lasso procedure to focus on managing the false-positive rate when selecting confounders for inclusion in a multivariable model. The double-selection technique carried out by the Lasso can be used for other population-based studies with a large imbalance between the number of features and observations (i.e. high-dimensional data). It provided an objective means of selecting important covariates, rather than relying on a P value in a stepwise selection process. The double-selection technique provided robust estimates of the treatment effects in the presence of large number potential confounding variables, and is proved to be a useful tool for analysis of complex population-level survival data.
We also observed that adjustment for the time dependence of the treatment effect was important in the setting of rapid change in clinical practice as was the case for melanoma in this era. In a clinical trial setting, the two treatment groups are randomized and the subsequent lines of treatment may be more homogeneous or predictable. However, in this population-based study, patients in different treatment groups are from three different eras, where access to subsequent lines of treatment and type of treatment varied. The model, utilizing time-dependent treatment variables, took into account switches between treatments, and was thus important in the context of our study. Notably, the HR estimates are similar to the phase III trial results (e.g. CA184-024 first-line study). Because the duration and timing of ipilimumab varied between patients and took place over multiple lines of drug treatment and eras, it is not surprising that accounting for these dynamics led to more stable point estimates.
Our study has several limitations. First, our estimates may have been influenced by residual confounding, particularly for unobserved variables. The burden of disease was unknown for our metastatic patients, although the ipilimumab treatment effect was observed to be present across subgroups in trials, and followed similar trends in subgroups in our population-based cohort [2,3]. We also note that no patients in our population started with other agents concurrently with ipilimumab as in the seminal trials, although the efficacy of these agents (dacarbazine, gp100 vaccine) is thought to be limited in this setting [2,3]. Many of the earliest treated ipilimumab patients in Ontario received chemotherapy before immunotherapy. This may conceivably have influenced results. However, we found that the observed effect of ipilimumab ( < 60 days chemotherapy) was similar to both the results from trials involving heavily pretreated melanoma and previously untreated melanoma [2,3]. Moreover, subgroup analysis limited to ipilimumab patients with and without previous chemotherapy produced similar results. This suggests that the exposure to less than 60 days chemotherapy probably had little to no impact on the estimated treatment effect.
Overall, our findings suggest that the improved survival with first-line ipilimumab observed in the randomized controlled trial setting can be reproduced at the population level. Moreover, ipilimumab was associated with improved survival across subgroups considered in our population sample. Finally, we note the importance of time-dependent adjustment in our complex sample, and the value of machine learning techniques in selecting important confounding variables in complex administrative data sets.
T.P. Hanna holds a research chair and received pilot funding provided by the Ontario Institute for Cancer Research through funding provided by the Government of Ontario ( # IA-035). This study was supported by a Canadian Institutes of Health Research Operating Grant (MOP 137022).
This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results, and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
Parts of this material are based on data and information provided by Cancer Care Ontario (CCO). The opinions, results, view, and conclusions reported in this paper are those of the authors and do not necessarily reflect those of CCO. No endorsement by CCO is intended or should be inferred.
Parts of this material are based on data and information compiled and provided by CIHI. However, the analyses, conclusions, opinions and statements expressed herein are those of the authors, and not necessarily those of CIHI.
The authors thank IMS Brogan Inc. for use of their Drug Information Database.
Authors’ contributions: All authors have made substantive contributions to the study, according to the Credit Taxonomy.
Conflicts of interest
There are no conflicts of interest.
1. Pardoll DM. The blockade of immune checkpoints in cancer immunotherapy. Nat Rev Cancer. 2012; 12: 252–264
2. Hodi FS, O’Day SJ, McDermott DF, Weber RW, Sosman JA, Haanen JB, et al. Improved survival with ipilimumab in patients with metastatic melanoma. N Engl J Med. 2010; 363: 711–723
3. Robert C, Thomas L, Bondarenko I, O’Day S, Weber J, Garbe C, et al. Ipilimumab plus dacarbazine for previously untreated metastatic melanoma. N Engl J Med. 2011; 364: 2517–2526
4. Schadendorf D, Hodi FS, Robert C, Weber JS, Margolin K, Hamid O, et al. Pooled analysis of long-term survival data from phase II and phase III trials of ipilimumab in unresectable or metastatic melanoma. J Clin Oncol. 2015; 33: 1889–1894
5. Gogas HJ, Kirkwood JM, Sondak VK. Chemotherapy for metastatic melanoma: time for a change? Cancer. 2007; 109: 455–464
6. Uprety D, Bista A, Chennamadhavuni A, Niroula A, Jafri SIM, Smith A, et al. Survival trends among patients with metastatic melanoma in the pretargeted and the post-targeted era: a US population-based study. Melanoma Res. 2018; 28: 56–60
7. Hanna TP, Nguyen P, Baetz T, Booth CM, Eisenhauer E. A population-based study of survival impact of new targeted and immune-based therapies for metastatic or unresectable melanoma. Clin Oncol (R Coll Radiol). 2018; 30: 609–617
8. Donia M, Kimper-Karl ML, Hoyer KL, Bastholt L, Schmidt H, Svane IM. The majority of patients with metastatic melanoma are not represented in pivotal phase III immunotherapy trials. Eur J Cancer. 2017; 74: 89–95
9. Robles SC, Marrett LD, Clarke EA, Risch HA. An application of capture-recapture methods to the estimation of completeness of cancer registration. J Clin Epidemiol. 1988; 41: 495–501
10. Tran JM, Schwartz R, Fung K, Rochon P, Chan AW. Comprehensive capture of cutaneous melanoma by the Ontario Cancer Registry: validation study using community pathology reports. Cancer Causes Control. 2016; 27: 137–142
11. Kralj B; Measuring Rurality – RIO2008 BASIC: methodology and results. OMA economics department2009. Available at: http://www.eriestclairlhin.on.ca/Page.aspx?id=11606
. [Accessed 12 May 2017].
12. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998; 36: 8–27
13. Tibshirani R. Regression shrinkage and selection via the lasso. J R Stat Soc. 1996; 58: 267–288
14. Belloni A, Chernozhukov V, Wei Y. Post-selection inference for generalized linear models with many controls. J Bus Econ Stat. 2016; 34: 606–619
15. Maio M, Grob JJ, Aamdal S, Bondarenko I, Robert C, Thomas L, et al. Five-year survival rates for treatment-naive patients with advanced melanoma who received ipilimumab plus dacarbazine in a phase III trial. J Clin Oncol. 2015; 33: 1191–1196
16. Conforti F, Pala L, Bagnardi V, De Pas T, Martinetti M, Viale G, et al. Cancer immunotherapy efficacy and patients’ sex: a systematic review and meta-analysis. Lancet Oncol. 2018; 19: 737–746