The Image Biomarker Standardization Initiative: Standardized Convolutional Filters for Reproducible Radiomics and Enhanced Clinical Insights

Introduction

Radiomics involves the high-throughput extraction of quantitative features from medical images to support clinical decision making (1,2). Relatively few radiomics decision support tools have entered the clinic because their clinical translation is restricted by both the lack of standardization of the extraction process and by lack of quality clinical evidence for their efficacy (3). Focusing on software-related aspects of the extraction process, the Image Biomarker Standardization Initiative (IBSI) previously established modality-independent standards for digital image processing and computation of handcrafted, quantitative radiomic features (4). This improved reproducibility and interchangeability of IBSI-compliant radiomics software packages, provided that the extraction process is configured the same between packages (5,6).

Filters (Table) are frequently used in radiomics analyses to enhance and quantify potentially clinically relevant characteristics and textures in medical images, such as the peritumoral region, blood vessels, contrast agent uptake, degree of calcification, or fibrosis, among others (7) (Appendix S1). For example, Beuque et al (8) applied a Laplacian of Gaussian filter to contrast-enhanced mammography to classify breast lesions into benign and malignant cases. The Laplacian of Gaussian filter enhanced the regions with contrast agent uptake, amplifying the signal, and therefore was found to be important for classifying lesion malignancy. Many filters, including the Laplacian of Gaussian filter used by Beuque et al, rely on convolution. Convolution is a mathematical operation where a filter (here an array of numbers) is systematically slid across the entire image (Fig 1). This process yields a filtered image that enhances and spatially locates potentially relevant image characteristics. However, the computational implementation of these filters has not been standardized, and quantitative features extracted from regions of interest in the filtered images were found to be poorly reproducible between radiomics software packages (9) (Fig 2). Consequently, radiomics decision support tools that incorporate features computed from regions of interest inside filtered images may be difficult to reproduce, validate, and translate clinically.

Glossary of Terms

Figure 1:

Download as PowerPoint

Figure 2:

Download as PowerPoint

Because convolutional filters are both important and commonly used, the IBSI aimed to improve reproducibility of radiomics decision support tools involving these filters and to facilitate their clinical translation through a modality-independent software standardization process by establishing definitions for convolutional filters, including commonly used ones such as wavelets and Laplacian of Gaussian filters; by integrating convolutional filters into the previously established general radiomics image processing scheme (4); and by providing data sets, associated reference filtered images, reference feature values, and tools for verification and calibration of radiomics software packages.

Materials and Methods

Study Design

This standardization effort was divided into three phases (Fig 3) and was conducted between September 2020 and December 2022. During the first two phases the implementation and use of convolutional filters were standardized. Phase 1 concerned the creation of reference filtered images (ie, the expected result of applying a convolutional filter with specific parameters to an image). In phase 2, convolutional filters were integrated into a radiomics workflow for the purpose of finding reference values for radiomic features computed from filtered images. In phase 3, we assessed whether standardization of convolutional filters resulted in reproducible feature values. A website (https://ibsi.radiomics.hevs.ch/; Appendix S2) was created to coordinate the study.

Figure 3:

Download as PowerPoint

Results

First Phase Results

Of the 36 filtered images that were assessed in the first phase, moderate or better consensus was found for 17 (47%) at the initial point (Fig 4). At the final point, moderate or better consensus was achieved for 33 (92%) configurations, of which 24 (67%) were very strong. Thus, 33 reference filtered images were established. Full consensus was reached for configurations corresponding to mean filters, Laplacian of Gaussian filters, Laws kernels, Gabor kernels, and separable and nonseparable wavelets (including decomposed forms). Weak or no consensus was achieved for three (8%) configurations, corresponding to configurations involving Riesz transformations (Fig S1).

Figure 4:

Download as PowerPoint

Discussion

Convolutional filters enhance specific structures and patterns in medical images and are commonly used in radiomics analyses. However, because of a lack of proper consensus-based reference implementations, features computed from filtered images provided by these filters were difficult to reproduce (9). In our study, 15 teams from seven countries collaborated to remedy this situation by providing reference filtered images, reference feature values, and reference documentation. We were able to standardize and validate eight different filter types: mean, Laplacian of Gaussian, Laws and Gabor kernels, and separable and nonseparable wavelet filters in both undecomposed and decomposed forms. Thirty-three reference filtered images and 323 reference feature values, computed from filtered images, were established to standardize radiomics analyses across various imaging modalities.

Our results complement the previous results of the IBSI (4). The IBSI focused on standardizing both the image processing scheme for radiomics and a large set of radiomic features. It aimed to improve reproducibility of radiomics studies by mitigating the effect of using different radiomics software packages and by providing a common framework for describing methodologic details. This study adds to the previous work by standardizing the use of convolutional filters frequently used in radiomics.

Despite the overall success of the standardization process, there were two instances in which we did not achieve the desired level of success. First, we were unable to standardize Riesz transformations that, despite their attractive characteristics from a signal processing perspective, were not easy to implement. Thus, too few teams contributed data for Riesz transformations, and we could not establish their reference filtered images and reference values. Because Riesz transformations are rarely used in radiomics studies, the impact should be minimal. Second, several features could not always be computed in a reproducible manner, notably the coefficient of variation and quartile coefficient features in conjunction with high- and band-pass convolutional filters. Such filters are characterized by a filtered image with a mean intensity of zero. In the presence of high- and band-pass convolutional filters, the mathematical division operation present in both features led to otherwise negligible numeric differences between teams becoming relevant, resulting in poor reproducibility. Therefore, these features should not be used in combination with high- and band-pass filters.

Our work has several implications. First, we found that reproducible implementation of most types of convolutional filters across different radiomics software is not straightforward, as evidenced by the initial lack of consensus on reference filtered images in phase 1 (Appendix S8). Therefore, we assume that existing clinical or research radiomics software, which incorporates convolutional filters in advanced image analysis workstations, may yield feature values that are not externally reproducible. This might impede external validation and subsequent clinical translation until the software is made to be compliant.

The second implication is that software labeled as IBSI-compliant is expected to reproduce the reference filtered images and reference feature values found in our study, insofar as convolutional filters are available in the software, in addition to the existing reference feature values (4). Developers of radiomics software supporting convolutional filters should aim to make their software compliant to improve reproducibility of radiomics analyses and allow for translation of enhanced clinical insights offered by convolutional filters. Developers should then clearly label their software as IBSI-compliant, to make it easier for users to identify and use their software for research and/or clinical purposes (with regulatory approval). Compliance may be checked using website-based tools (https://ibsi.radiomics.hevs.ch/), or by manually comparing the produced filtered images and feature values against the provided reference data. Compliant software is expected to produce filtered images on which every voxel deviates from the reference filtered image by at most 1% of the range of intensity values of the reference filtered image (Appendix S5). Similarly, feature values must be within the specified tolerance margin around their reference feature values.

The third implication is that even though we contextualized our efforts within radiologic imaging, our work is relevant for quantitative image analysis in general, including digital pathology. Like our previous study (4), this study is anticipated to improve reproducibility of radiomics analyses beyond the modalities (digital phantoms, chest CT) and settings (non–small cell lung cancer) examined during the initial two phases of this study. To provide supporting preliminary evidence, we conducted validation using a publicly available data set composed of patients with soft-tissue sarcoma and multiple imaging modalities. The outcomes of the validation phase reinforce the potential applicability of our work in diverse settings.

Our study had limitations. First, its scope was restricted. Compliance with IBSI reference values helps to improve reproducibility of radiomic features (5,6). However, the results of a radiomics analysis also depend on image acquisition, reconstruction, segmentation, and data analysis steps (18,19), which we did not address here or in our previous work. Differences in, for example, image acquisition protocols are known to affect the appearance of an image, and therefore also reproducibility of radiomic features (20). Such effects can be reduced by harmonization and cross-calibration of scanners and protocols (21) and post hoc techniques such as perturbation (22,23), batch normalization (24), and other methods (25). Second, participation in the IBSI does not guarantee that a particular software package is compliant with the IBSI reference standard. Changes introduced in software (5) or design choices may limit compliance (26). Third, we standardized intensity-based statistical features computed from filtered images but no other types of features. Morphologic features are mostly redundant because these are based on segmentation masks that are explicitly not altered by convolutional filtering. Most texture features, in our estimation, would be too abstract to allow for interpretation when computed from filtered images. Their use may add hundreds or thousands of features to a radiomics analysis, which complicates the process of creating generalizable and interpretable radiomics models in the typical setting where at most a few hundred images are available for analysis. Finally, the IBSI has focused on radiomics using handcrafted features, and with this work offers a comprehensive reference standard for their computation. However, we recognize that there are more features and other filters than the ones we have standardized so far. These are not implemented often and will be hard to standardize for that reason.

In conclusion, we standardized eight types of convolutional filters for radiomics to ensure that the enhanced clinical insights that can be gained through their use can be validated and reproduced. Going forward, developers should ensure compliance of their software with the proposed reference standards, and users are encouraged to use compliant software. A web-based tool is available for compliance checking. In the future, the Image Biomarker Standardization Initiative will focus on deep learning applications of radiomics, with the goal to provide reference standards for image preprocessing.