Can Delta Radiomics Improve the Prediction of Best Overall Response, Progression-Free Survival, and Overall Survival of Melanoma Patients Treated with Immune Checkpoint Inhibitors?
by
, , , , and
Felix Peisen
1,*,
Annika Gerken
2,
Alessa Hering
2,3,
Isabel Dahm
1,
Konstantin Nikolaou
1,4,
Sergios Gatidis
1,5,
Thomas K. Eigentler
6,7,
Teresa Amaral
6,
Jan H. Moltz
2,† and
Ahmed E. Othman
8,† 1
Department of Diagnostic and Interventional Radiology, Eberhard Karls University, Tuebingen University Hospital, Hoppe-Seyler-Straße 3, 72076 Tuebingen, Germany
2
Fraunhofer Institute for Digital Medicine MEVIS, Max-von-Laue-Straße 2, 28359 Bremen, Germany
3
Diagnostic Image Analysis Group, Radboudumc, Geert Grooteplein Zuid 10, 6525 GA Nijmegen, The Netherlands
4
Cluster of Excellence iFIT (EXC 2180) “Image-Guided and Functionally Instructed Tumor Therapies”, Faculty of Medicine, Eberhard Karls University, 72076 Tuebingen, Germany
5
Max Planck Institute for Intelligent Systems, Max-Planck-Ring 4, 72076 Tuebingen, Germany
6
Center of Dermato-Oncology, Department of Dermatology, Eberhard Karls University, Tuebingen University Hospital, Liebermeisterstraße 25, 72076 Tuebingen, Germany
7
Department of Dermatology, Venereology and Allergology, Charité—Universitätsmedizin Berlin, Freie Universität Berlin and Humboldt-Universität zu Berlin, Luisenstraße 2, 10117 Berlin, Germany
8
Institute of Neuroradiology, Johannes Gutenberg University Hospital Mainz, Langenbeckstraße 1, 55131 Mainz, Germany
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Cancers 2024, 16(15), 2669; https://doi.org/10.3390/cancers16152669
Submission received: 17 June 2024
/
Revised: 16 July 2024
/
Accepted: 26 July 2024
/
Published: 26 July 2024
(This article belongs to the Special Issue Cancer Biomarkers—Detection and Evaluation of Response to Therapy)
Abstract
:Simple Summary
The incidence of metastatic melanoma is rising, making it imperative to identify patients who do not benefit from immunotherapy. This study aimed to develop a radiomic biomarker, using segmentations from 146 baseline and 146 first follow-up CT scans, to predict best overall response, progression-free survival, and overall survival across various immunotherapies. We volumetrically segmented the total tumour load, excluding cerebral metastases. This study also examined whether reducing the number of segmented metastases per patient affects predictive accuracy. The findings suggest that delta radiomics could enhance the prediction of best overall response, progression-free survival, and overall survival in metastatic melanoma patients undergoing first-line immunotherapy. Although volumetric whole tumour load segmentation is complex, it may provide predictive benefits.
Abstract
Background: The prevalence of metastatic melanoma is increasing, necessitating the identification of patients who do not benefit from immunotherapy. This study aimed to develop a radiomic biomarker based on the segmentation of all metastases at baseline and the first follow-up CT for the endpoints best overall response (BOR), progression-free survival (PFS), and overall survival (OS), encompassing various immunotherapies. Additionally, this study investigated whether reducing the number of segmented metastases per patient affects predictive capacity. Methods: The total tumour load, excluding cerebral metastases, from 146 baseline and 146 first follow-up CTs of melanoma patients treated with first-line immunotherapy was volumetrically segmented. Twenty-one random forest models were trained and compared for the endpoints BOR; PFS at 6, 9, and 12 months; and OS at 6, 9, and 12 months, using as input either only clinical parameters, whole-tumour-load delta radiomics plus clinical parameters, or delta radiomics from the largest ten metastases plus clinical parameters. Results: The whole-tumour-load delta radiomics model performed best for BOR (AUC 0.81); PFS at 6, 9, and 12 months (AUC 0.82, 0.80, and 0.77); and OS at 6 months (AUC 0.74). The model using delta radiomics from the largest ten metastases performed best for OS at 9 and 12 months (AUC 0.71 and 0.75). Although the radiomic models were numerically superior to the clinical model, statistical significance was not reached. Conclusions: The findings indicate that delta radiomics may offer additional value for predicting BOR, PFS, and OS in metastatic melanoma patients undergoing first-line immunotherapy. Despite its complexity, volumetric whole-tumour-load segmentation could be advantageous.
1. Introduction
Melanoma, particularly in its metastatic stage IV form, is increasingly diagnosed [1,2]. Since the advent of immune checkpoint inhibitors like PD-1 (programmed death-1) inhibitors (nivolumab and pembrolizumab), and CTLA-4 (cytotoxic T-lymphocyte-associated protein 4) inhibitors (ipilimumab), prognoses have improved significantly [3,4,5,6,7]. However, treatment resistance and severe side effects such as autoimmune pancreatitis and pneumonitis pose significant challenges [8,9,10]. Therefore, there is a critical need to identify patients who do not benefit from these therapies to guide them towards more effective alternatives. Radiomics, which extracts large datasets from imaging studies and translates them into biomarkers, has shown promise in this regard [11,12,13]. Initial radiomic applications include pre-treatment imaging with CT (computed tomography) or 18F-FDG-PET/CT (flourine-18 fluorodeoxyglucose positron emission tomography/computed tomography). Some studies have demonstrated the predictive power of baseline CT radiomics for survival in melanoma patients [14,15,16,17] although their results have been inconsistent [18].
Delta radiomics, which evaluates changes in imaging features over time, offers an advanced approach. This method combines baseline and follow-up imaging data to generate predictive biomarkers [19,20,21]. Previous studies have shown the potential of delta radiomics in predicting outcomes such as overall survival and progression-free survival in small cohorts of melanoma patients. Guerrisi et al. and Wang et al. performed pilot studies in small groups of patients with malignant melanoma and reported that CT delta texture analysis predicted overall survival and progression-free survival as well as early response to immunotherapy and pseudo progression [21,22]. Dercle et al. studied a larger cohort undergoing immunotherapy with pembrolizumab. They followed a whole-tumour-load segmentation approach on baseline and first follow-up CTs and published a radiomic signature for predicting overall survival. However, the reported time required for manual lesion segmentation of approximately one minute per lesion per scan, and the restriction to pembrolizumab monotherapy limits the clinical application of the signature [23].
This study aimed to evaluate whether a more generalizable radiomic biomarker could be developed for the endpoints BOR, PFS, and OS, using a larger and more diverse patient sample. Additionally, it explored whether reducing the number of segmented metastases affects predictive performance.
2. Materials and Methods
2.1. Patients
This study included patients with stage IV malignant melanoma (AJCC 8th edition [24]) treated between 2015 and 2018, as recorded in the local dermatology melanoma registry. Patients treated with first-line immunotherapy (PD-1 checkpoint inhibitor monotherapy or combination of a PD-1 checkpoint inhibitor and a CTLA-4 checkpoint inhibitor), who had available contrast-enhanced baseline and first follow-up CT imaging and measurable disease at baseline were included. The study protocol received institutional review board approval (protocol code 092/2019BO2, 21 February 2019), and informed consent was waived due to the retrospective design. A workflow diagram is shown in Figure 1.
2.2. Imaging
Baseline and first follow-up CTs were retrieved from the local picture archiving and communication system (PACS), anonymised, and uploaded into custom software (SATORI, Fraunhofer MEVIS, Bremen, Germany) for volumetric segmentation of all measurable metastases. A radiologist (F.P.) with six years of oncologic imaging experience conducted manual segmentation for baseline CTs in consensus reading with A.E.O. and S.G. (both specialists in oncologic imaging), while follow-up CT segmentations were precomputed by an algorithm trained on baseline segmentations [25] and reviewed by the same radiologists. A detailed distribution of the CT scanners and imaging parameters can be found in Table S1 in Supplementary Materials. Examples of different timepoint responses can be depicted from Figure S1. Radiomic feature extraction was performed using the Pyradiomics Python package (v3.1.0) [11], and delta features were computed at a patient level. A detailed description of the radiomic feature extraction and aggregation is provided in Supplementary Materials.
2.3. Model Development
Random forest models were trained for seven different clinical endpoints (best overall response to therapy according to Response Evaluation Criteria In Solid Tumors (RECIST) 1.1 criteria [26] (binarised: complete or partial response = response; stable or progressive disease = no response); progression-free survival after six, nine, and twelve months; and overall survival after six, nine, and twelve months. Model development and validation were conducted using Python, version 3.6.13 (Python Software Foundation, Beaverton, OR, USA).
2.4. Validation
Considering the conclusions of Kocak et al. [27], model performance was validated against a clinical parameter-only model that used the following features as input: age, gender, type of immunotherapy, localization of primary tumour, histological subtype of primary tumour, BRAF (v-Raf murine sarcoma viral oncogene homolog B1) V600E mutation status, baseline lactate dehydrogenase level, follow-up lactate dehydrogenase level, baseline S100 level, follow-up S100 level, number of metastatic organs in baseline CT, and presence of cerebral metastases or hepatic metastases. Tests were also conducted against a model using only the ten largest metastases per patient for the extraction of radiomic features, simulating a more realistic segmentation approach from a clinical perspective. Performance was estimated using ten-time-repeated five-fold cross-validation. For a detailed description of the radiomics feature extraction and aggregation, the machine learning model, and the model evaluation, see Supplementary Materials S2.1–S2.3.
2.5. Statistical Analysis
Analyses were conducted using Excel, version 2019 (Microsoft Corporation, Redmond, DC, USA), SPSS Statistics 29 (IBM, Armonk, NY, USA), and R, version 3.6.2 (R Program for Statistical Computing, Vienna, Austria). The area under the curve (AUC) of the receiver-operating-characteristic (ROC) curve was used as a classification performance metric. Statistically significant superior performance of the extended model was achieved if the 95% confidence intervals (CI) of the mean AUC of the baseline and extended models did not overlap. Significant predictive capacity of a model following the outcome distribution was achieved if the lower bound of the CI was higher than 0.5.
3. Results
3.1. Patients’ Characteristics
The final cohort consisted of 146 patients, predominantly male (63%), with a median age of 66 years. The most common histological subtype was nodular melanoma (28%). Most patients received either nivolumab and ipilimumab combination therapy (45%) or pembrolizumab monotherapy (42%). A detailed description of the patients’ characteristics is shown in Table 1.
3.2. Random Forest Models for Binarised Best Overall Therapy Response
Three random forest models were cross-validated for binarised BOR (see Figure 2). The model using clinical data and whole-tumour burden radiomics achieved the highest AUC (0.81), followed by the model using clinical data and radiomics from the ten largest metastases (AUC 0.79). The clinical data-only model achieved an AUC of 0.75. Detailed values are shown in Table 2. As the confidence intervals of the clinical and radiomics models overlapped, statistical significance was not reached according to our definition.
3.3. Random Forest Models for Progression-Free Survival
Nine models were cross-validated for PFS at 6, 9, and 12 months (Figure 3 shows the endpoint PFS at 12 months). The best performance was achieved with models combining clinical data and radiomics from the whole-tumour burden (AUCs 0.82, 0.80, and 0.77 for 6, 9, and 12 months, respectively). Detailed values are shown in Table 2. As the confidence intervals of the clinical and radiomics models overlapped, statistical significance was not reached according to our definition.
3.4. Random Forest Models for Overall Survival
Nine models were cross-validated for OS at 6, 9, and 12 months (Figure 4 shows the endpoint OS at 12 months). The model for OS at 6 months performed best with whole-tumour-burden radiomics (AUC 0.74). Models for OS at 9 and 12 months performed best with radiomics from the ten largest metastases (AUC 0.71 and 0.75, respectively). Detailed values are shown in Table 2. As the confidence intervals of the clinical and radiomics models overlapped, statistical significance was not reached according to our definition.
4. Discussion
Radiomics offers promising potential for identifying melanoma patients who may not benefit from immunotherapy. Several studies have reported that relevant features can be identified from baseline CT imaging. The delta approach may further enhance predictive capabilities of radiomics, as demonstrated by data published by Dercle et al. [23]. However, the proposed algorithm lacks clinical transferability as it is limited to pembrolizumab therapy and requires segmentation of the entire tumour burden on baseline and follow-up imaging. Therefore, our aim was to develop a model applicable to a wider range of immunotherapies and to investigate whether the quantity of lesions required for segmentation could be reduced to a more manageable number.
This study demonstrated that delta radiomics of the whole-tumour volume improved model performance for most endpoints (BOR; PFS 6, 9, 12 months; and OS 6 months). The AUCs were all numerically superior compared to a model using only clinical features. Compared to the results published by Dercle et al. [23], our AUC values for the prediction of OS were lower (0.74–0.70 compared to 0.92). The reason for this discrepancy is most likely the composition of the investigated samples studied. Dercle et al. restricted their cohort to melanoma patients treated with a single agent programmed cell death 1 blocker (pembrolizumab). In contrast, we aimed for a more generalizable approach and included patients with pembrolizumab or nivolumab monotherapy or combined nivolumab/ipilimumab therapy to allow for improved clinical applicability. The positive effect of delta radiomics was still present in our approach, but it should be noted that the confidence intervals for all endpoints overlapped. Therefore, according to our definition, there was a numerical improvement, but not a statistically significant difference.
Manual volumetric segmentation of the whole-tumour burden is very time-consuming. At approximately one minute per lesion per scan, a patient with multifocal metastatic melanoma and only thirty metastases will require an hour to segment a baseline and follow-up scan. This approach is highly problematic in terms of clinical application. Reducing the number of segmented lesions to the ten largest metastases still provided valuable information but with slightly lower predictive performance, except for OS 9 and 12 months. This shows that segmentation of the entire tumour burden, although time-consuming, may provide more information. Advances in AI-enhanced segmentation may mitigate these challenges, making volumetric segmentation more feasible in clinical settings. Several techniques for automated or semi-automated lesion segmentation have been reported [25,28,29,30]. Hering et al. published two studies that proved the feasibility of automated tracking and volumetric segmentation using the example of soft tissue and lymph node metastases. They could show that their proposed pipeline with a so called initial “one-click” segmentation at baseline imaging and subsequent automated segmentation in follow-up imaging is non-inferior to manual segmentation, significantly saves time, and reduces inter-reader variability [25,30]. The pipeline is based on the nnU-Net by Isensee et al., a deep learning framework, that showed impressive results at segmentation tasks, using a self configurating architecture [29]. Moawad et al. nicely discussed the limitations of the U-Net architecture for 3D medical imaging segmentation and potential solutions, such as variants that accept 3D imaging as input (V-Net, 3D U-Net, hybrid Dense U-Net) or different architectures such as fully convolutional networks (FCN) [28]. Although not all these methods have become established in daily routine image reading, segmentation of the entire tumour burden in a manageable amount of time is theoretically no longer unrealistic and might enable radiologists to access additional information, such as radiomics or volumetric RECIST [31,32].
Our study has strengths and limitations. We used a registry with prospective data documentation to identify the sample studied. The cohort contained a large number of patients treated according to current guidelines at a tertiary referral centre. Volumetric segmentation was carried out by an experienced radiologist in consensus reading with two experts in oncological imaging. Prospective validation on an external dataset and a second segmentation by a second reader are lacking. A larger sample size would have been beneficial, and we are hoping to expand our dataset in the future.
5. Conclusions
Overall, delta radiomics shows potential for improving the prediction of BOR, PFS, and OS in metastatic melanoma patients receiving first-line immunotherapy. Despite its complexity, a volumetric segmentation of the whole-tumour burden could be favourable. Further research with larger cohorts and prospective validation is needed to confirm these findings and enhance clinical applicability.
Supplementary Materials
The following supporting information can be downloaded at https://www.mdpi.com/article/10.3390/cancers16152669/s1. Scan parameters and CT vendor details; Detailed description of the radiomic feature extraction and aggregation, the machine learning model, and model evaluation; Table S1: CT scanners and vendors. Figure S1: Examples of different timepoint responses: A, C, E show baseline CT imaging of three different patients. B, D, F show CT imaging of the first follow-up with timepoint responses “partial response”, “stable disease” and “progressive disease”, respectively.
Author Contributions
Conceptualization, F.P., A.G., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Methodology, F.P., A.G., A.H., I.D., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Software, F.P., A.G., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Validation, F.P., A.G., A.H., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Formal analysis, F.P., A.G., J.H.M. and A.E.O.; Investigation, F.P., A.G., A.H., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Resources, F.P., A.G., A.H., I.D., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Data curation, F.P., A.G., A.H., I.D., T.K.E., T.A., J.H.M. and A.E.O.; Writing—original draft preparation, F.P., J.H.M. and A.E.O.; Writing—review and editing, F.P., A.G., A.H., I.D., K.N., S.G., T.K.E., T.A., J.H.M. and A.E.O.; Visualization, F.P. and J.H.M.; Supervision, S.G., T.K.E., J.H.M. and A.E.O.; Project administration, F.P., A.G., J.H.M. and A.E.O.; Funding acquisition, F.P., S.G., T.K.E., T.A., J.H.M. and A.E.O. All authors have read and agreed to the published version of the manuscript.
Funding
This study was funded by the SPP2177 program of the German Research Foundation (Deutsche Forschungsgemeinschaft, ‘DFG’), project number #428216905. The DFG was not involved in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Institutional Review Board Statement
This study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Ethics Committee of the Medical Faculty Eberhard-Karls-University Tuebingen (protocol code 092/2019BO2, 21 February 2019).
Informed Consent Statement
Patient consent was waived by the Institutional Ethics Committee due to the retrospective study design.
Data Availability Statement
The complete dataset cannot be made freely available to the public, due to privacy regulations. A minimal dataset, codes, and materials used in this study may be made available for purposes of reproducing or extending the analysis, pending material transfer agreements, after a reasonable and well justified request to Felix Peisen.
Acknowledgments
We acknowledge support from the Open Access Publishing Fund of the University of Tübingen.
Conflicts of Interest
The authors declare that they have no competing interests.
Abbreviations
| ALM | acral lentiginous melanoma |
| AUC | area under the curve |
| BRAF | v-Raf murine sarcoma viral oncogene homolog B1 |
| CI | confidence interval |
| CR | complete response |
| CT | computed tomography |
| CTLA-4 | cytotoxic T-lymphocyte-associated protein 4 |
| CV | cross-validation |
| IQR | interquartile range |
| LDH | lactate dehydrogenase |
| LMM | lentigo maligna melanoma |
| n | number |
| NM | nodular melanoma |
| OS | overall survival |
| PACS | picture archiving and communication system |
| PD | progressive disease |
| PD-1 | programmed death 1 |
| PET | positron emission tomography |
| PFS | progression-free survival |
| PR | partial response |
| RECIST | Response Evaluation Criteria In Solid Tumors |
| ROC | receiver operating characteristic |
| SD | stable disease |
| SSM | superficial spreading melanoma |
References
- Whiteman, D.C.; Green, A.C.; Olsen, C.M. The Growing Burden of Invasive Melanoma: Projections of Incidence Rates and Numbers of New Cases in Six Susceptible Populations through 2031. J. Investig. Dermatol. 2016, 136, 1161–1171. [Google Scholar] [CrossRef]
- Schadendorf, D.; van Akkooi, A.C.J.; Berking, C.; Griewank, K.G.; Gutzmer, R.; Hauschild, A.; Stang, A.; Roesch, A.; Ugurel, S. Melanoma. Lancet 2018, 392, 971–984. [Google Scholar] [CrossRef]
- Sussman, T.A.; Knackstedt, R.; Wei, W.; Funchain, P.; Gastman, B.R. Outcomes of stage IV melanoma in the era of immunotherapy: A National Cancer Database (NCDB) analysis from 2014 to 2016. J. Immunother. Cancer 2022, 10, 4994. [Google Scholar] [CrossRef]
- Conforti, F.; Pala, L.; Bagnardi, V.; De Pas, T.; Martinetti, M.; Viale, G.; Gelber, R.D.; Goldhirsch, A. Cancer immunotherapy efficacy and patients’ sex: A systematic review and meta-analysis. Lancet Oncol. 2018, 19, 737–746. [Google Scholar] [CrossRef]
- Weiss, S.A.; Wolchok, J.D.; Sznol, M. Immunotherapy of Melanoma: Facts and Hopes. Clin. Cancer Res. 2019, 25, 5191–5201. [Google Scholar] [CrossRef]
- Knight, A.; Karapetyan, L.; Kirkwood, J.M. Immunotherapy in Melanoma: Recent Advances and Future Directions. Cancers 2023, 15, 1106. [Google Scholar] [CrossRef]
- Ralli, M.; Botticelli, A.; Visconti, I.C.; Angeletti, D.; Fiore, M.; Marchetti, P.; Lambiase, A.; de Vincentiis, M.; Greco, A. Immunotherapy in the Treatment of Metastatic Melanoma: Current Knowledge and Future Directions. J. Immunol. Res. 2020, 2020, 9235638. [Google Scholar] [CrossRef]
- Kozar, I.; Margue, C.; Rothengatter, S.; Haan, C.; Kreis, S. Many ways to resistance: How melanoma cells evade targeted therapies. Biochim. Biophys. Acta Rev. Cancer 2019, 1871, 313–322. [Google Scholar] [CrossRef]
- Ostrowski, S.M.; Fisher, D.E. Biology of Melanoma. Hematol./Oncol. Clin. North. Am. 2021, 35, 29–56. [Google Scholar] [CrossRef]
- Eikenes, G.; Liszkay, G.; Balatoni, T.; Czirbesz, K.; Hunyadi, K.; Kozéki, Z.; Kispál, M.T.; Baranyai, F.; Danyi, T.; Bőcs, K.; et al. Therapeutic and Adverse Effect of Anti-PD1 Immunotherapy in Melanoma: A Retrospective, Single-Institute Study of 222 Patients. Cancers 2023, 15, 3966. [Google Scholar] [CrossRef]
- van Griethuysen, J.J.M.; Fedorov, A.; Parmar, C.; Hosny, A.; Aucoin, N.; Narayan, V.; Beets-Tan, R.G.H.; Fillion-Robin, J.C.; Pieper, S.; Aerts, H. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Res. 2017, 77, e104–e107. [Google Scholar] [CrossRef]
- McGale, J.; Hama, J.; Yeh, R.; Vercellino, L.; Sun, R.; Lopci, E.; Ammari, S.; Dercle, L. Artificial Intelligence and Radiomics: Clinical Applications for Patients with Advanced Melanoma Treated with Immunotherapy. Diagnostics 2023, 13, 3065. [Google Scholar] [CrossRef]
- Fusco, R.; Granata, V.; Simonetti, I.; Setola, S.V.; Iasevoli, M.A.D.; Tovecci, F.; Lamanna, C.M.P.; Izzo, F.; Pecori, B.; Petrillo, A. An Informative Review of Radiomics Studies on Cancer Imaging: The Main Findings, Challenges and Limitations of the Methodologies. Curr. Oncol. 2024, 31, 403–424. [Google Scholar] [CrossRef]
- Durot, C.; Mule, S.; Soyer, P.; Marchal, A.; Grange, F.; Hoeffel, C. Metastatic melanoma: Pretreatment contrast-enhanced CT texture parameters as predictive biomarkers of survival in patients treated with pembrolizumab. Eur. Radiol. 2019, 29, 3183–3191. [Google Scholar] [CrossRef]
- Trebeschi, S.; Drago, S.G.; Birkbak, N.J.; Kurilova, I.; Calin, A.M.; Delli Pizzi, A.; Lalezari, F.; Lambregts, D.M.J.; Rohaan, M.W.; Parmar, C.; et al. Predicting response to cancer immunotherapy using noninvasive radiomic biomarkers. Ann. Oncol. 2019, 30, 998–1004. [Google Scholar] [CrossRef]
- Flaus, A.; Habouzit, V.; de Leiris, N.; Vuillez, J.-P.; Leccia, M.-T.; Simonson, M.; Perrot, J.-L.; Cachin, F.; Prevot, N. Outcome prediction at patient level derived from pre-treatment 18F-FDG PET due to machine learning in metastatic melanoma treated with anti-PD1 treatment. Diagnostics 2022, 12, 388. [Google Scholar] [CrossRef]
- Ungan, G.; Lavandier, A.-F.; Rouanet, J.; Hordonneau, C.; Chauveau, B.; Pereira, B.; Boyer, L.; Garcier, J.-M.; Mansard, S.; Bartoli, A.; et al. Metastatic melanoma treated by immunotherapy: Discovering prognostic markers from radiomics analysis of pretreatment CT with feature selection and classification. Int. J. Comput. Assist. Radiol. Surg. 2022, 17, 1867–1877. [Google Scholar] [CrossRef]
- Peisen, F.; Hansch, A.; Hering, A.; Brendlin, A.S.; Afat, S.; Nikolaou, K.; Gatidis, S.; Eigentler, T.; Amaral, T.; Moltz, J.H.; et al. Combination of Whole-Body Baseline CT Radiomics and Clinical Parameters to Predict Response and Survival in a Stage-IV Melanoma Cohort Undergoing Immunotherapy. Cancers 2022, 14, 2992. [Google Scholar] [CrossRef]
- Abbas, E.; Fanni, S.C.; Bandini, C.; Francischello, R.; Febi, M.; Aghakhanyan, G.; Ambrosini, I.; Faggioni, L.; Cioni, D.; Lencioni, R.A.; et al. Delta-radiomics in cancer immunotherapy response prediction: A systematic review. Eur. J. Radiol. Open 2023, 11, 100511. [Google Scholar] [CrossRef]
- Nardone, V.; Reginelli, A.; Grassi, R.; Boldrini, L.; Vacca, G.; D’Ippolito, E.; Annunziata, S.; Farchione, A.; Belfiore, M.P.; Desideri, I.; et al. Delta radiomics: A systematic review. La Radiol. Medica 2021, 126, 1571–1583. [Google Scholar] [CrossRef]
- Wang, Z.L.; Mao, L.L.; Zhou, Z.G.; Si, L.; Zhu, H.T.; Chen, X.; Zhou, M.J.; Sun, Y.S.; Guo, J. Pilot Study of CT-Based Radiomics Model for Early Evaluation of Response to Immunotherapy in Patients With Metastatic Melanoma. Front. Oncol. 2020, 10, 1524. [Google Scholar] [CrossRef]
- Guerrisi, A.; Russillo, M.; Loi, E.; Ganeshan, B.; Ungania, S.; Desiderio, F.; Bruzzaniti, V.; Falcone, I.; Renna, D.; Ferraresi, V.; et al. Exploring CT Texture Parameters as Predictive and Response Imaging Biomarkers of Survival in Patients With Metastatic Melanoma Treated With PD-1 Inhibitor Nivolumab: A Pilot Study Using a Delta-Radiomics Approach. Front. Oncol. 2021, 11, 704607. [Google Scholar] [CrossRef]
- Dercle, L.; Zhao, B.; Gonen, M.; Moskowitz, C.S.; Firas, A.; Beylergil, V.; Connors, D.E.; Yang, H.; Lu, L.; Fojo, T.; et al. Early Readout on Overall Survival of Patients With Melanoma Treated With Immunotherapy Using a Novel Imaging Analysis. JAMA Oncol. 2022, 8, 385–392. [Google Scholar] [CrossRef]
- Gershenwald, J.E.; Scolyer, R.A. Melanoma Staging: American Joint Committee on Cancer (AJCC) 8th Edition and Beyond. Ann. Surg. Oncol. 2018, 25, 2105–2110. [Google Scholar] [CrossRef]
- Hering, A.; Peisen, F.; Amaral, T.; Gatidis, S.; Eigentler, T.; Othman, A.; Moltz, J.H. Whole-Body Soft-Tissue Lesion Tracking and Segmentation in Longitudinal CT Imaging Studies. Med. Imaging Deep. Learn. 2021, 25, 312–326. [Google Scholar]
- Eisenhauer, E.A.; Therasse, P.; Bogaerts, J.; Schwartz, L.H.; Sargent, D.; Ford, R.; Dancey, J.; Arbuck, S.; Gwyther, S.; Mooney, M.; et al. New response evaluation criteria in solid tumours: Revised RECIST guideline (version 1.1). Eur. J. Cancer 2009, 45, 228–247. [Google Scholar] [CrossRef]
- Kocak, B.; Bulut, E.; Bayrak, O.N.; Okumus, A.A.; Altun, O.; Borekci Arvas, Z.; Kavukoglu, I. NEgatiVE results in Radiomics research (NEVER): A meta-research study of publication bias in leading radiology journals. Eur. J. Radiol. 2023, 163, 110830. [Google Scholar] [CrossRef]
- Moawad, A.W.; Fuentes, D.T.; ElBanan, M.G.; Shalaby, A.S.; Guccione, J.; Kamel, S.; Jensen, C.T.; Elsayes, K.M. Artificial Intelligence in Diagnostic Radiology: Where Do We Stand, Challenges, and Opportunities. J. Comput. Assist. Tomogr. 2022, 46, 78–90. [Google Scholar] [CrossRef]
- Isensee, F.; Jaeger, P.F.; Kohl, S.A.A.; Petersen, J.; Maier-Hein, K.H. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nat. Methods 2021, 18, 203–211. [Google Scholar] [CrossRef]
- Hering, A.; Westphal, M.; Gerken, A.; Almansour, H.; Maurer, M.; Geisler, B.; Kohlbrandt, T.; Eigentler, T.; Amaral, T.; Lessmann, N.; et al. Improving assessment of lesions in longitudinal CT scans: A bi-institutional reader study on an AI-assisted registration and volumetric segmentation workflow. Int. J. Comput. Assist. Radiol. Surg. 2024, 30, 1–9. [Google Scholar] [CrossRef]
- Winter, K.S.; Hofmann, F.O.; Thierfelder, K.M.; Holch, J.W.; Hesse, N.; Baumann, A.B.; Modest, D.P.; Stintzing, S.; Heinemann, V.; Ricke, J.; et al. Towards volumetric thresholds in RECIST 1.1: Therapeutic response assessment in hepatic metastases. Eur. Radiol. 2018, 28, 4839–4848. [Google Scholar] [CrossRef] [PubMed]
- Hayes, S.A.; Pietanza, M.C.; O’Driscoll, D.; Zheng, J.; Moskowitz, C.S.; Kris, M.G.; Ginsberg, M.S. Comparison of CT volumetric measurement with RECIST response in patients with lung cancer. Eur. J. Radiol. 2016, 85, 524–533. [Google Scholar] [CrossRef] [PubMed]
Figure 1.
Workflow diagram.
Figure 2.
AUCs for the prediction of binarised best overall response. Left graph represents the model using only clinical parameters; middle graph represents the model using clinical parameters plus radiomic features from all metastases per patient; right graph represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Figure 2.
AUCs for the prediction of binarised best overall response. Left graph represents the model using only clinical parameters; middle graph represents the model using clinical parameters plus radiomic features from all metastases per patient; right graph represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Figure 3.
AUCs for the prediction of progression-free survival at twelve months. (Left graph) represents the model using only clinical parameters; (middle graph) represents the model using clinical parameters plus radiomic features from all metastases per patient; (right graph) represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Figure 3.
AUCs for the prediction of progression-free survival at twelve months. (Left graph) represents the model using only clinical parameters; (middle graph) represents the model using clinical parameters plus radiomic features from all metastases per patient; (right graph) represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Figure 4.
AUC curves for the prediction of overall survival for twelve months. (Left graph) represents the model using only clinical parameters; (middle graph) represents the model using clinical parameters plus radiomic features from all metastases per patient; (right graph) represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Figure 4.
AUC curves for the prediction of overall survival for twelve months. (Left graph) represents the model using only clinical parameters; (middle graph) represents the model using clinical parameters plus radiomic features from all metastases per patient; (right graph) represents the model using clinical parameters plus radiomic features from the largest ten metastases per patient.
Table 1.
Patient’s characteristics.
| Clinical Data | ||
|---|---|---|
| Age (years) [median, (IQR)] | 66 (22%) | |
| Gender (male) [n, %] | 92 (63%) | |
| Localization of primary tumour [n, %] | head/neck | 34 (23%) |
| torso | 35 (24%) | |
| upper extremity | 21 (14%) | |
| lower extremity | 38 (26%) | |
| other | 7 (5%) | |
| n/a | 11 (8%) | |
| Histological subtype [n, %] | SSM | 35 (24%) |
| NM | 41 (28%) | |
| LMM | 8 (5%) | |
| ALM | 22 (14%) | |
| mucosal | 7 (5%) | |
| occult | 9 (6%) | |
| n/a | 27 (18%) | |
| BRAF V600E mutation status [n, %] | BRAF wildtype | 101 (69%) |
| BRAF mutation | 41 (28%) | |
| n/a | 4 (3%) | |
| Baseline LDH [n, %] | normal (<250 U/L) | 31 (21%) |
| elevated (≥250 U/L) | 100 (69%) | |
| n/a | 15 (10%) | |
| FU1 LDH [n, %] | normal (<250 U/L) | 86 (59%) |
| elevated (≥250 U/L) | 57 (39%) | |
| n/a | 3 (2%) | |
| Baseline S100B [n, %] | normal (<0.1 µg/L) | 77 (53%) |
| elevated (≥0.1 µg/L) | 59 (40%) | |
| n/a | 10 (7%) | |
| FU1 S100B [n, %] | normal (<0.1 µg/L) | 80 (55%) |
| elevated (≥0.1 µg/L) | 61 (42%) | |
| n/a | 5 (3%) | |
| Number of metastatic organs [n, %] | 1–3 | 132 (90%) |
| > 3 | 14 (1%) | |
| Presence of cerebral metastases [n, %] | 23 (16%) | |
| Presence of hepatic metastases [n, %] | 39 (27%) | |
| Therapy [n, %] | pembrolizumab | 61 (42%) |
| nivolumab | 19 (13%) | |
| nivolumab + ipilimumab | 66 (45%) | |
| Baseline CT lesion counts [n] | all | 3188 |
| lung | 1411 | |
| liver | 584 | |
| soft tissue/skin | 416 | |
| lymph nodes | 478 | |
| skeletal | 77 | |
| spleen | 74 | |
| other | 148 | |
| FU1 CT lesion counts [n] | all | 4836 |
| lung | 2104 | |
| liver | 1083 | |
| soft tissue/skin | 707 | |
| lymph nodes | 588 | |
| skeletal | 71 | |
| spleen | 92 | |
| other | 191 | |
| Patient outcome | ||
| Best overall response (RECIST 1.1) [n, %] | CR | 26 (18%) |
| PR | 46 (31%) | |
| SD | 22 (15%) | |
| PD | 48 (33%) | |
| n/a | 4 (3%) | |
| Progression-free survival for 6 months [n, %] | yes | 68 (44%) |
| no | 66 (48%) | |
| n/a | 12 (8%) | |
| Progression-free survival for 9 months [n, %] | yes | 54 (37%) |
| no | 73 (50%) | |
| n/a | 19 (13%) | |
| Progression-free survival for 12 months [n, %] | yes | 41 (28%) |
| no | 77 (53%) | |
| n/a | 28 (19%) | |
| Overall survival after 6 months [n, %] | yes | 110 (75%) |
| no | 19 (13%) | |
| n/a | 17 (12%) | |
| Overall survival after 9 months [n, %] | yes | 90 (62%) |
| no | 27 (18%) | |
| n/a | 29 (20%) | |
| Overall survival after 12 months [n, %] | yes | 70 (48%) |
| no | 30 (21%) | |
| n/a | 46 (31%) | |
Abbreviations: ALM, acral lentiginous melanoma; CR, complete response;; FU1, first follow-up, IQR, interquartile range; LDH, lactate dehydrogenase; LMM, lentigo maligna melanoma; n/a, not available; NM, nodular melanoma;; PD, progressive disease;, PR, partial response; RECIST, Response Evaluation Criteria In Solid Tumors; SD, stable disease; SSM, superficial spreading melanoma.
Table 2.
Number of cases for the different endpoints with class distributions and mean AUC from a 10 × 5-fold CV and 95% confidence interval computed by bootstrapping the 10 × 5-fold CV. Class 0 for BOR = PD/SD; class 1 for BOR = PR/CR; class 0 for PFS/OS = no; class 1 for PFS/OS = yes.
Table 2.
Number of cases for the different endpoints with class distributions and mean AUC from a 10 × 5-fold CV and 95% confidence interval computed by bootstrapping the 10 × 5-fold CV. Class 0 for BOR = PD/SD; class 1 for BOR = PR/CR; class 0 for PFS/OS = no; class 1 for PFS/OS = yes.
| Binary Endpoint | Cases n (Class 0 + 1) | Model with Clinical Features Only. AUC (95%CI) | Model with Clinical Features + Whole- Tumour-Load Radiomic Features. AUC (95%CI) | Model with Clinical Features + Radiomic Features from Largest Ten Lesions. AUC (95%CI) |
|---|---|---|---|---|
| Best overall response | 142 (70 + 72) | 0.750 (0.672, 0.822) | 0.811 (0.745, 0.876) | 0.794 (0.726, 0.862) |
| PFS 6 months | 134 (66 + 68) | 0.797 (0.726, 0.859) | 0.824 (0.756, 0.882) | 0.814 (0.747, 0.874) |
| PFS 9 months | 127 (73 + 54) | 0.764 (0.684, 0.832) | 0.797 (0.730, 0.855) | 0.774 (0.702, 0.841) |
| PFS 12 months | 118 (77 + 41) | 0.742 (0.658, 0.816) | 0.769 (0.698, 0.839) | 0.741 (0.667, 0.815) |
| OS 6 months | 129 (19 + 110) | 0.721 (0.588, 0.848) | 0.742 (0.598, 0.870) | 0.718 (0.583, 0.852) |
| OS 9 months | 117 (27 + 90) | 0.684 (0.568, 0.788) | 0.704 (0.594, 0.808) | 0.708 (0.590, 0.811) |
| OS 12 months | 101 (31 + 70) | 0.724 (0.617, 0.822) | 0.744 (0.642, 0.836) | 0.746 (0.641, 0.838) |
Abbreviations: AUC, area under the curve; BOR, best overall response; CI, confidence interval; CR, complete response; CV, cross-validation; n, number; OS, overall survival; PD, progressive disease; PFS, progression-free survival; PR, partial response; SD, stable disease.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
MDPI and ACS Style
Peisen, F.; Gerken, A.; Hering, A.; Dahm, I.; Nikolaou, K.; Gatidis, S.; Eigentler, T.K.; Amaral, T.; Moltz, J.H.; Othman, A.E. Can Delta Radiomics Improve the Prediction of Best Overall Response, Progression-Free Survival, and Overall Survival of Melanoma Patients Treated with Immune Checkpoint Inhibitors? Cancers 2024, 16, 2669. https://doi.org/10.3390/cancers16152669
AMA Style
Peisen F, Gerken A, Hering A, Dahm I, Nikolaou K, Gatidis S, Eigentler TK, Amaral T, Moltz JH, Othman AE. Can Delta Radiomics Improve the Prediction of Best Overall Response, Progression-Free Survival, and Overall Survival of Melanoma Patients Treated with Immune Checkpoint Inhibitors? Cancers. 2024; 16(15):2669. https://doi.org/10.3390/cancers16152669
Chicago/Turabian StylePeisen, Felix, Annika Gerken, Alessa Hering, Isabel Dahm, Konstantin Nikolaou, Sergios Gatidis, Thomas K. Eigentler, Teresa Amaral, Jan H. Moltz, and Ahmed E. Othman. 2024. "Can Delta Radiomics Improve the Prediction of Best Overall Response, Progression-Free Survival, and Overall Survival of Melanoma Patients Treated with Immune Checkpoint Inhibitors?" Cancers 16, no. 15: 2669. https://doi.org/10.3390/cancers16152669
Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.
