Using Machine Learning to Predict Response to Image-guided Therapies for Hepatocellular Carcinoma

Introduction

Interventional oncology, a subspecialty of interventional radiology, offers various image-guided therapies for the treatment of hepatocellular carcinoma (HCC), including percutaneous ablation, transarterial chemoembolization (TACE), and transarterial radioembolization (TARE). While these treatments have proven efficacious in improving HCC survival rates, treatment responses vary due to differences in baseline tumor burden, tumor biology, and procedural factors (Fig 1). There is an unmet need to triage patients to the most effective therapies and optimize treatment sequencing.

Figure 1:

Download as PowerPoint

Over the past decade, multiple treatment algorithms and staging systems, such as the Barcelona Clinic Liver Cancer (BCLC) staging system, the Hong Kong Liver Cancer (HKLC) staging system, and the Pre-TACE-Predict model, have been established for patient selection (1–5). However, these models, which rely largely on limited clinical, laboratory, and qualitative imaging features, demonstrate only modest performance in external test sets (6–8). With recent advances in image processing, researchers have turned to image-based methods, such as radiomics and artificial intelligence (AI), for the prediction of HCC treatment response. This review will discuss the state of AI applications in HCC prognostication.

Introduction to AI and Machine Learning: Modeling Steps

AI is a general term that describes the use of technology to mimic human intelligence (Fig 2) (9,10). Machine learning (ML) is a branch of AI in which a model learns from training data and makes effective predictions by teaching itself. For simplicity, we will focus on two main categories of ML: supervised and unsupervised learning. In supervised learning, the programmer directs the learning process by showing the model correct examples. For example, in HCC treatment response analysis, the radiologist first identifies the malignant lesion on pretreatment images, segments the lesion section by section, and reviews posttreatment imaging to determine treatment response. In this example, the labels for the training data are “response” and “nonresponse,” and the inputs are segmented pretreatment lesion images.

Figure 2:

Download as PowerPoint

In contrast, unsupervised ML works independently to discover patterns within unlabeled data sets. Revisiting the example above, instead of defining treatment labels, investigators can input pre- and posttreatment images and let the algorithm form clusters, elucidating trends that the investigator had not set out to find. This can simultaneously be a strength and a weakness as the algorithm may group the posttreatment lesions as intended into different treatment groups based on pre- and posttreatment lesion size and viability, or it could group by background noise. Due to the unsupervised nature of the process, this type of ML is more prone to biases when trained on nonrepresentative samples. Compared with supervised ML, which is resource intensive due to the need to create labeled data sets, unsupervised learning algorithms do not require manual data labeling.

ML is often coupled with radiomics: the high-throughput mining of quantitative features from image voxel data. Radiologist-defined semantic features are qualitative (ie, visually assessed) imaging features such as lesion size and arterial phase hyperenhancement, whereas radiomic features provide a comprehensive quantification of a lesion's features and include things like shape, intensity, and texture. Coupling ML with quantitative imaging features yields further insight into the complex relationships between radiomic variables and clinical outcomes (11). The processing of imaging features using ML can be broadly categorized into two pathways: conventional ML radiomics analysis and deep learning (DL).

Conventional ML Radiomics Analysis

The conventional ML radiomics workflow is a stepwise process that uses image data to produce models that aid in clinical decision-making (Fig 3). The pipeline begins with image segmentation: manual, semiautomatic, or automatic. In manual segmentation, radiologists contour pretreatment lesions on every section. In semiautomatic segmentation, radiologists identify lesions of interest and set thresholding parameters that best capture them. The program automatically captures the volume of interest, which radiologists then fine-tune. Finally, in automatic segmentation, a DL model previously trained to segment tumors processes raw images and automatically delineates the volume of interest. Automatic segmentation is not only time and resource efficient but also has better accuracy in predicting treatment response (12,13).

Figure 3:

Download as PowerPoint

During image processing, image quality is improved by applying filters that assist in noise reduction while maintaining crisp lesion edges, after which pixel and voxel features are quantified during feature extraction. Features can be manually selected based on the researcher's prior experience (ie, handcrafted features), automatically extracted through radiomics packages (eg, PyRadiomics) (14), or derived from the second-to-last layer of a neural network (ie, learned features). While learned features can adapt to training data and thus generalize well to unseen data, handcrafted features are set from the start and limited by expert knowledge (15). Next, feature selection and dimension reduction are performed by selecting the most generalizable features to prevent model overfitting (ie, models that work well only for the images on which they were trained). Finally, during model building, highly predictive variables are selected using filter, wrapper, or embedded methods to build robust models (Table 1) (16). The last steps of feature selection and model building constitute an iterative process where different features and algorithms are trialed to achieve the most predictive model.

Table 1: Steps for Conventional Machine Learning Radiomics Analysis

Prediction of HCC Treatment Response Using AI

A central challenge of HCC treatment is accurately determining which patients will respond best to which treatment or treatment combinations. A comprehensive literature review was performed using electronic databases including PubMed, MEDLINE, and Embase. Keywords included combinations of “HCC”; “ML” and/or “DL” and/or “AI”; “TACE” and/or “TARE” and/or “ablation”; and their variations. Only human studies published in English before December 2022 were included. Two authors (C.H. and A.L.) independently examined the titles and abstracts, and full texts when necessary, to determine eligible studies (Fig 4). The reference lists of eligible studies were screened to identify further eligible studies. Studies were excluded if they (a) included only qualitative imaging features and not quantitative imaging features, (b) did not use ML techniques in model construction, or (c) used postprocedural images for prediction of recurrence. Details of the included studies involving TACE and ablation are summarized in Tables 2 and 3, respectively. A total of 21 studies were included in the final analysis: 17 involved TACE, and four involved ablation. None of the included studies involved TARE. Study methodological quality was assessed using components of the Quality Assessment of Diagnostic Accuracy Studies (QUADAS) tool for AI (QUADAS-AI) (21), as well as QUADAS-2 (22) (Table S1).

Figure 4:

Download as PowerPoint

Table 2: Summary of Studies Using Conventional ML Radiomics Analysis or DL Modeling for Prognostication of TACE for Treatment of Hepatocellular Carcinoma

Table 3: Summary of Studies Using Conventional ML Radiomics Analysis or DL Modeling for Prognostication of Ablation for Treatment of Hepatocellular Carcinoma

TACE Studies

Included studies involving TACE (n = 17) were published between 2018 and 2022. Most studies (13 of 17, 76%) were conducted in China. Median cohort size for training and testing sets combined was 194 patients (range, 36–789; IQR, 101–371). The majority of studies (11 of 17, 65%) used CT while four (24%) used MRI, one (6%) used US, and one (6%) used digital subtraction angiography (DSA). Seven of 17 (41%) studies used ML techniques, five (29%) used DL techniques, and five (29%) used a combination of both. Modified Response Evaluation Criteria in Solid Tumors (mRECIST) was the most common standard used to assess treatment response (12 of 17 studies, 71%). Six of 17 (35%) studies had training and validation sets, whereas 11 (65%) had separate training and testing sets. Of the 11 studies with separate training and testing sets, three (27%) performed internal testing and eight (73%) performed both internal and external testing of the models.

Seven domains were assessed regarding bias and applicability for each study using components of the QUADAS-AI and QUADAS-2 scores shown in Table S1. Of 119 domains in total for TACE articles, 29 domains were at high risk: 15 for risk of bias and 14 for applicability concerns. Risk of bias stemmed from inappropriate patient selection for six studies (these studies did not use Barcelona Clinic Liver Cancer staging for assessing TACE eligibility) and from index test issues for nine studies (these studies did not perform external testing). Applicability concerns stemmed primarily from patient selection, as most of these models were trained and tested on Chinese populations, where HCC etiology differs from that in the United States.

For most studies, the primary goal was predicting whether patients would respond to TACE based on pre-TACE images. Some studies incorporated survival analysis to determine whether nonresponders had poorer prognosis than TACE responders and might instead benefit from other treatments, such as ablation or systemic therapy combined with TACE (19,23).

Unlike most studies that used pretreatment CT or MRI scans, Zhang et al (19) used DL to develop an automatic tumor segmentation model that integrated clinical variables and preprocedural DSA videos of superior mesenteric and hepatic arteries to predict treatment response or nonresponse to TACE. The authors demonstrated that there was a significant difference in 3-year progression-free survival between responders and nonresponders as classified with the DL model. The authors introduced an innovative approach to determining TACE eligibility using DSA videos. DSA offers a way to directly visualize the arterial supply of target tumors that can be challenging to visualize with pretreatment CT or MRI. However, DSA image quality is limited by motion artifacts, and small, hypovascular lesions are challenging to visualize.

Peng et al (24) developed a conventional ML model to predict initial response to TACE treatment based on pre-TACE CT images. The authors demonstrated that patients predicted to be treatment responders had longer progression-free survival and overall survival than patients predicted to be treatment nonresponders. Of note, the authors then applied the model to 46 patients with HCC from The Cancer Genome Atlas (TCGA) database to analyze differential gene expression across the two cohorts and to explore potential model mechanisms. This approach of incorporating genetic data was made possible with ML, highlighting its versatility when processing large volumes of data.

Image-guided Thermal Ablation Studies

We identified four ML studies dedicated to HCC prognostication after thermal ablation. Three of the four such studies (75%) used US or contrast-enhanced US (CEUS) to evaluate postablation prognosis. Two studies included microwave ablation, one included radiofrequency ablation, and one included both microwave and radiofrequency ablation. All studies had separate training and internal testing cohorts, but none included external testing.

Ma et al (20) compared the performance of a DL model trained on CEUS images with that of a conventional ML model trained on static US images in predicting late HCC recurrence. As expected, the CEUS model outperformed the static US image model, likely because CEUS not only shows morphologic features but also provides real-time dynamic blood perfusion information that is highly correlated with ablation success. Furthermore, to unmask the “black box” nature of the models (ie, lack of transparency), the authors attempted to visualize the model's selected features. They found that the model highlighted certain frames of the CEUS cine that reflected lesion wash-in and washout periods. Study limitations included the lack of standardization of US and CEUS examinations, which were largely operator dependent and involved different machines, parameters, and contrast agent doses and timing (25).

Liu et al (26) used clinical data and imaging features extracted from CEUS images not only to predict 2-year progression-free survival in patients with early-stage HCC treated with radiofrequency ablation and surgical resection, but also to investigate the optimization of treatment selection. The authors concluded that 17.3% and 27.3% of patients who underwent radiofrequency ablation and surgical resection, respectively, would have had a better outcome if they had received the other treatment instead. Even though the study sample was limited to patients with early-stage HCC and a multicenter prospective study is necessary to test the clinical significance of these findings, the authors’ approach to optimizing treatment selection furthers clinical applicability.

TARE Studies

Data regarding ML-based prognostication of tumor response following TARE remain limited. There were no TARE studies that met the inclusion and exclusion criteria of this review. Several studies, however, have used non-ML radiomics models to predict prognosis (27–30).

DL-driven radiation dosimetry models are currently being studied in other cancers for predicting the biodistribution of theragnostic pairs, with the goal of customizing therapy dose based on unique patient characteristics (31). It would be ideal to be able to predict response to TARE using preprocedural imaging alone: While our review did not identify any studies with this approach, some studies have used both baseline images and initial postprocedural images to predict future response (28,29). Wei et al (29) built a multivariate logistic regression model using radiomic features and absorbed dose metrics extracted from post-TARE yttrium 90 PET scans to predict TARE response at first follow-up. Importantly, the mean absorbed dose alone achieved similar predictive ability, suggesting that the actual distribution and dose of the microspheres is a bigger determinant of TARE results than intratumoral hemodynamics extrapolated from preprocedural baseline imaging. Though these results do not assist in patient selection for initial therapy, being able to predict response at an early stage could be helpful in determining when earlier retreatment would be indicated. Limitations included a small sample size (30 patients, 105 lesions) as PET or CT posttherapy imaging is not routinely performed at every institution.

Barriers to Clinical Implementation of Prognostication Models

Though image-based AI models for prognostication of HCC treatment response are promising, several advances are required before widespread clinical use. Just as with medical devices, AI software should be subjected to rigorous approval and regulatory processes to ensure safe and effective patient care. Recently, the U.S. Food and Drug Administration proposed a pathway for clearing ML software as a medical device (32). In this section, we discuss elements of this pathway in the context of AI-based HCC prognostication models.

Conclusion

Image-based artificial intelligence (AI) models predict the performance of interventional treatment for hepatocellular carcinoma (HCC) by automatically extracting quantitative imaging features and analyzing the complex relationships between these features and clinical outcomes. Despite the active landscape, there are no applications of AI models approved by the U.S. Food and Drug Administration for image-guided treatments of HCC or interventional radiology overall. This lack of AI models approved by the U.S. Food and Drug Administration highlights the need for clinical validation beyond the development of models (49). Though many obstacles stand before widespread implementation for clinical use, current applications have shown the potential of predictive AI as a tool for personalizing treatment in patients with HCC.