Pancreatic neuroendocrine tumors (PanNETs) are rare tumors of the pancreas that account for approximately 3% of all pancreatic tumors.1 In recent years, there has been an increase in the incidence of these tumors, a finding that can be attributed to increased utilization of cross-sectional imaging.2,3,4 These tumors display heterogeneous histopathological and clinical behaviors ranging from being indolent to highly aggressive.5,6

Nodal disease and tumor grade have been identified as two of the strongest predictors of biological behavior and outcomes in PanNETs.7,8,9,10,11,12 On the basis of these findings, guidelines have been established for patient management.13,14,15 Overall, these guidelines recommend that PanNETs that are functional, symptomatic, G2 or G3, or larger than > 2 cm should undergo resection.13,14,15 For patients with asymptomatic, small (< 2 cm), low-grade nonfunctional (NF)-PanNETs, the current guidelines recommend surveillance and potential resection upon changes in tumor characteristics.14 However, recent studies have shown that ~ 15% of patients in this cohort have poor biological characteristics and can rapidly progress to metastatic disease.5 Therefore, to improve the current management of these patients, it is important to predict the presence of nodal disease preoperatively. In the past, predictive models based on tumor size and grade have been developed, however, they are not well validated.16,17,18,19

Radiomics is the study of extracting and analyzing multiple datapoints obtained from radiological imaging to quantitatively characterize lesions and identify predictors of clinicopathological characteristics and patient outcomes.20 In recent years, it has been increasingly applied to develop tools for precision approaches for multiple tumor types including pancreatic neoplasms.21,21,22 Literature on applications of radiomics to PanNETs is limited, with the majority of studies applying radiomics as a tool to assess a wide variety of tumor characteristics and limited focus on addressing specific clinical questions.20 Currently, only a few studies have applied radiomic analyses to assess nodal disease among PanNETs, with varying results.16,17,18

The aim of the current study was to assess the role of radiomics in determining nodal status in patients with NF-PanNETs. In particular, we combined radiomic features with clinicopathological characteristics to develop a model that could accurately predict nodal disease in patients with PanNETs.

Patients and Methods

Patient Selection

An Institutional Review Board (IRB)-approved and Health Insurance Portability and Accountability Act (HIPAA)-compliant single-center retrospective study was performed using a prospectively maintained institutional registry at Johns Hopkins Hospital (Baltimore, MD). We identified patients who underwent resection for well-differentiated (G1 or G2) NF-PanNETs between 2003 and 2022 for whom in-house pancreas protocol computed tomography (PPCT), performed within 3 months of resection, was available (Fig. 1). Patients with functional PanNETs, poorly differentiated tumors, metastatic disease at diagnosis, prior pancreatic surgery, or those receiving neoadjuvant therapy, were excluded. Additionally, patients in whom nodal disease was not assessed on histopathological examination were excluded.

Fig. 1
figure 1

Flowchart of patient selection criteria

Full size image

Histopathological Assessment

Histopathological analysis was conducted on all surgical specimens. As routine, samples were assessed for tumor size, tumor grade, presence of lymphovascular and perineural invasion, and nodal status. Tumor grade was determined on the basis of the 2017 World Health Organization (WHO) classification.23 Patients who were managed prior to 2017 were only included if their tumor grade had been assessed retrospectively.

Image Acquisition

Patients with PanNETs were scanned on a dual-source multidetector computed tomography (MDCT) scanner (Somatom Definition, Definition Flash, or Force, Siemens Healthineers), 64-slice MDCT scanner (Somatom Sensation 64, Siemens Healthineers), or 16-slice MDCT scanner (Somatom Sensation 16, Siemens Healthineers). Patients were injected with 100–120 mL of iohexol (Omnipaque, GE Healthcare) at an injection rate of 4–5 mL/s. Scanning protocols were customized for each patient to minimize dose but were on the order of 120 kVp, 270 effective mAs, and 0.6–0.8 pitch. The collimation was 128 × 0.6 mm or 192 × 0.6 mm for the dual-source scanner and 64 × 0.6 mm for the 64-MDCT scanner.

All CT scans included both arterial and venous phase images according to institutional protocol. Arterial phase imaging was performed with bolus triggering, between approximately 25 s and 30 s after injection, while venous phase imaging was performed at 60–70 s after injection. All images were reconstructed into 0.75 mm slice thickness and 0.5 mm slice intervals for segmentation and radiomics analysis because thin slices are less susceptible to volume averaging and can potentially detect more subtle differences in the pattern of voxel intensities. Radiomics analysis was performed on both arterial and venous phase images.

Image Segmentation

The whole three-dimensional (3D) volume of the entire pancreas including the tumor, background pancreas, major peripancreatic vessels, and intrapancreatic common bile duct were manually segmented by four trained researchers (3–5 years of experience) using commercial segmentation software (Velocity 3.2.0, Varian Medical Systems) (Fig. 2). The contours were verified by three abdominal radiologists with 7–30 years of experience. Cases were randomly and evenly assigned to the researchers and radiologists. Although inter-reader agreement was not evaluated, prior research has demonstrated that interobserver variability generated from manual segmentation has minimal bearing on radiomics feature extraction and performance.24 The region of interest (ROI), consisting of the tumor, was used to extract quantitative tumor characteristics such as tumor size, location, shape, and attenuation.

Fig. 2
figure 2

Manually segmented pancreas (yellow), tumor (red), veins (blue), arteries (pink), and common bile duct (green) displayed in A axial and B coronal planes

Full size image

Image Analysis and Radiomic Features

A total of 2436 radiomics features from the segmented ROI volume were extracted using the open source Pyradiomics python package to express PanNETs on the basis of both venous and arterial phase images for each patient. For each scanning phase, 1218 radiomics features were used in this study and included 18 first-order statistics of the volumetric CT intensities, 14 shape features of the target structure, 68 texture features from a gray-level co-occurrence matrix, a gray-level run-length matrix, a gray-level size zone matrix and a gray-level dependence matrix, 688 texture features from the 8 filtered volumes by wavelets, and an additional 430 texture features from the filtered volumes by Laplacian of Gaussian (LoG).25

Statistical Analysis

Clinicopathological characteristics were represented as frequencies and percentages for the categorical variables and median and IQRs for continuous variables. For the continuous variables, a Mann–Whitney U test or Student’s t-test was used as appropriate, whereas for the categorical variables, the Pearson’s chi-squared test or Fisher exact test were used. A p-value of less than 0.05 was considered statistically significant. All statistical analyses were performed using STATA version 17.1 (StataCorp, College Station, TX).

A random forest model was trained and validated by combining 10 radiomic features selected from 2436 radiomic features via the feature selection method and clinicopathologic characteristics of the pancreatic tumor.26 The model’s ability to accurately predict the presence of nodal disease was evaluated using the C-statistic. Area under the curve (AUC), sensitivity, and specificity were reported. Repeated twofold cross validation of the model was subsequently conducted to optimize the model’s parameters. More specifically, we randomly split our whole cohort into half as training and half as validation to train and validate our models. This twofold cross-validation process was repeated, and our final model is an ensemble of all models during the repeat.

The study was approved by the Institutional Review Board for Human Research at our institution and complied with all Health Insurance Portability and Accountability Act regulations.

Results

Study Population

A total of 320 patients underwent surgical resection for PanNETs. Clinicopathological characteristics of these patients are presented in Table 1. Mean age was 59.4 ± 11.6 years and a majority were male (N = 173, 54.1%). On histopathological examination of the surgically resected specimen, the mean tumor size was 3.0 cm ± 2.3 cm. Nodal disease was present in 92 (28.8%) of patients. The mean number of nodes harvested per patient was 16.9 ± 7.9. Most patients had well-differentiated grade 1 (G1) tumors (N = 232, 72.5%), while the rest were grade 2 (G2).

Table 1 Clinicopathological characteristics of patients with small PanNETs stratified by presence of nodal disease
Full size table

Radiomic Feature-Based Prediction of Nodal Metastasis

A total of 2436 features were extracted from both the arterial and venous phases. Upon application of max-dependency, max-relevance, and min-redundancy, ten radiomic features were selected for inclusion in the final model (Supplementary Table 1).26 The ten radiomics features comprised five wavelet features and one LoG feature from venous phase while two wavelet features and two LoG features came from arterial phase.

Following this, a multivariate analysis was performed using logistic regression. Factors that were found to be associated with the presence of nodal disease included tumor size based on surgical pathology and grade (all p values < 0.05). A model based on these clinical factors was developed and an AUC of 0.77 was observed (Fig. 3). These factors were then incorporated with the RFs to develop a final prediction model. Upon internal validation of this model, a slight increase in the area under the curve (AUC) was observed with an AUC of 0.80 (Fig. 4). In the subset of patients with PanNETs smaller than 2 cm, the model correctly predicted nodal involvement in 85.0% of patients.

Fig. 3
figure 3

ROC for prediction of nodal positivity by the clinical model

Full size image
Fig. 4
figure 4

ROC for prediction of nodal positivity by the combined radiomics–clinical model

Full size image

The only factor that was found to be significantly associated with accurate prediction of nodal disease in these patients was tumor size, with a lower likelihood of accurate prediction with increasing tumor size (OR 0.87, 95% CI 0.79–0.96, p = 0.007) (Table 2). This could perhaps indicate a transition toward a more aggressive tumor biology as the tumor grows in size.

Table 2 Clinicopathological factors associated with accurate prediction of nodal disease
Full size table

Discussion

PanNETs display a wide range of biological behavior. One of the strongest factors found to be associated with patient outcomes is the presence of nodal disease.7,8,9,10,11,12 However, current guidelines only factor in tumor size and tumor grade for clinical decision-making. On the basis of the current guidelines, it is recommend that resection of PanNETs that are functional, symptomatic, G2 or G3, or those > 2 cm be performed.13,14,15 Considering the morbidity associated with pancreatic resections, it is recommended that patients with small low-grade PanNETs undergo surveillance.14 While these small PanNETs are indeed predominantly indolent in nature, approximately 15% exhibit malignant behavior, and studies have demonstrated improved long-term outcomes following resection.7 Individualizing patient management via identification of a unique subset of patients who are at a higher risk of progression is essential in aligning diverging evidence and establishing a consensus regarding their management. Nodal disease is among the strongest predictors of patient outcomes in PanNETs and if accurate preoperative prediction of nodal disease were possible, it would allow clinicians to factor this important aspect of disease biology in clinical decision-making prior to resection.8,9,10,11,12,13 The field of radiomics has recently emerged as a powerful tool to extract quantitative data from imaging, allowing for accurate prediction of clinicopathological characteristics and patient outcomes. Therefore, the aim of the current study was to develop a model on the basis of RFs combined with preoperatively available clinical factors to predict presence of nodal disease in PanNETs. We demonstrated that integration of radiomic features with clinicopathological features resulted in accurate prediction of nodal disease in patients with well-differentiated NF-PanNETs. The AUC achieved with radiomics and clinical factors was 0.80 as compared with 0.77 for clinical factors alone. While a substantial increase in AUC was not observed, these findings demonstrate that combination of clinical factors and radiomic features into multianalyte composite tools can result in incremental improvement of the predictive models. As future models are developed, potential addition of further variables could potentially improve this AUC to a greater extent.

In our study, the overall nodal positivity rate was 29% and nodal positivity rate for G1 tumors was 21%. This is in line with existing literature reporting nodal metastases in approximately 18–38% of patients with PanNETs with a median nodal positivity rate of 15.8% for G1 tumors.7,27,28 One of the challenges that are faced in considering nodal disease in the management of these patients is the inability of current imaging modalities to accurately predict the presence of nodal disease.29 A number of studies have previously explored risk stratification methods and developed risk scores and nomograms for predicting nodal metastases, incorporating tumor grade, size, site, and magnetic resonance imaging (MRI)-derived whole-tumor histogram analysis.19,30,31 Yet, preoperative determination of nodal involvement is still primarily dependent on the radiologist’s read, with CT being the most widely used preoperative staging modality for pancreatic neoplasms.32 While CT can accurately evaluate tumor size and vessel involvement, its diagnostic accuracy of nodal involvement is low owing to poor sensitivity (0–38%).29 The short axis nodal diameter of greater than 10 mm that many studies use as the criterion for lymph node metastasis has been demonstrated to have poor sensitivity (20–38%).33,34,35,36 Other imaging criteria for lymph node metastasis includes lymph node fusion, internal necrosis, ill-defined borders, nonuniform density, nonuniform enhancement, or involvement of the surrounding organs or blood vessels.37 These diagnostic criterion, however, are subjective and also yield suboptimal diagnostic accuracy.38 Despite ongoing improvements in imaging techniques, accurate preoperative staging of nodal status, therefore, remains a challenge.

Radiomics have previously demonstrated utility in preoperative assessment of a variety of tumor characteristics.39 Prior studies on applications of radiomics in preoperative prediction of nodal metastases have shown promising results in pancreatic ductal adenocarcinoma, colorectal cancer, bladder cancer, and lung adenocarcinoma.40,41,42,43 Most current literature on applications of radiomics in PanNETs has been on radiomics guided grading of tumors and have demonstrated strong results.44,45 In this study we developed a predictive model on the basis of radiomics features and clinical characteristics that was able to preoperatively predict nodal metastases in patients with PanNETs with good predictive capability. Notably, this model performed superiorly to non-radiomics-feature-based nodal metastases risk scores and nomograms previously reported in the literature (AUC range 0.622–0.71).19,30,31 While the role of radiomics in the evaluation of nodal metastases among PanNETs has been previously investigated, literature on the topic is limited, with only one CT-based study, one MR-based study, and one positron emission tomography (PET)/CT-based study on radiomics-guided prediction of nodal metastasis.16,17,18 The best performance reported to date in predicting nodal disease on the basis of radiomic features in PanNETs was by Mapelli et al., who demonstrated best in class correlation between two radiomic features (GrayLevelVariance and HighGrayLevelZoneEmphasis) extracted from T2 MRI and the presence of nodal metastases (AUC 0.992).16 Benedetti et al.18 conducted a comprehensive study correlating radiomics features extracted from pre- and post-contrast CT images with a variety of histopathological characteristics of PanNETs. In this study, they demonstrated that two individual radiomic features (pre-contrast CT neighborhood-intensity-difference and post-contrast CT intensity-size-zone) had adequate performance in predicting nodal metastases (AUC 0.75).18 Although both these studies demonstrated an association between RFs and nodal metastases, they did so on an individual radiomic feature level and did not combine multiple radiomic features and clinical features to create a unique radiomics signature as was done in our study. In another study using radiomics extracted from 68Ga-DOTA PET scans, a radiomics model was developed that significantly predicted lymph node involvement (p = 0.0151).17 This study, however, was limited by its small sample size of only 19 patients with nodal disease. While the aforementioned studies using MRI and PET/CT have shown promise, we were unable to establish direct comparisons between these imaging modalities in our study. At our institution, the most frequently used imaging modality for workup of pancreatic neoplasms is a CT, which was thus the modality we studied. It is possible that in the future, combining radiomics data extracted from multiple modalities may result in a more accurate prediction of nodal involvement.46

While previous studies on radiomics models capable of predicting nodal disease have demonstrated strong prediction ability, ours is the first large study to demonstrate that although radiomics do have fair utility in predicting nodal disease, results may not be as consistently robust as previously believed. Review of existing literature has previously demonstrated that up to 94% of published literature on radiomics has reported overwhelmingly positive findings.47 Despite the undeniable promise of radiomics, including highly favorable results from our own group in the past, these numbers raise concern of a potential publication bias within the field. Publication bias is well acknowledged across a variety of fields of research and is of particular concern in medical literature.48 Within the realm of radiomics, this matter is further complicated by the added ambiguity of the “black box” nature of radiomics methods, which makes reproducibility and independent validation of results across different institutions with varying imaging protocols challenging.49 This is highlighted in the lack of widespread prospective multicenter validation of current radiomics studies reporting equally strong results as reported within their primary cohorts. Furthermore, there currently exists a concerning lack of consistency between studies regarding the radiomics features associated with certain outcomes, with often little overlap between reported features across different studies, even when the same modality and cancer type are analyzed.50 In certain cases, identical radiomics features have also been paradoxically linked to positive outcomes in certain studies while simultaneously being linked with negative outcomes in other studies.51,52 While radiomics has demonstrated tremendous potential, considering these factors, it would be prudent to temper the unchecked expectations that the predominantly positive studies that have been published within the field have generated. To this end, dissemination of literature on radiomics that is methodologically sound and yet does not demonstrate highly accurate results, and perhaps even demonstrates negative results, is essential in contextualizing current trends in reporting.

A tangentially noteworthy finding in our study was that despite only a slight improvement upon integration of clinical factors, an improvement was observed nonetheless. This demonstrates that multiple factors contribute to the accurate prediction of nodal disease. While RF and clinical factors combined result in a promising tool, there is still need for improvement in the performance. Therefore, further novel biomarkers are required that could improve the performance of such models. In the past, various groups have investigated cfDNA, immunohistochemistry profiling, genetic mutations, and inflammatory markers, and have established their association with nodal disease.6,53,54,55,56 As these associations continue to emerge and are validated, integration of these novel variables with existing models has the potential to further improve current predictive capabilities and optimize risk stratification and surgical planning.

Our study does have several limitations that should be acknowledged. First, it was a retrospective single-center study with potential for bias. Second, radiomics features are known to be sensitive to alterations in image acquisition protocol and scanner type.57 Though we did validate our results in a separate cohort of patients, their scans were also conducted at our institution. Multicenter validation of our model with greater sample sizes is necessary before results may be generalized. Third, radiomics features were extracted from images that were manually segmented by a team of four researchers, creating the possibility of interobserver variability. All segmented scans, however, were reviewed for accuracy by experienced radiologists prior to inclusion. Previous research has also demonstrated that interobserver variability generated from manual segmentation has minimal bearing on RF extraction and performance.24 Fourth, of the study population, seven patients underwent enucleation and the nodal yield was low (number of harvested nodes 5, IQR 2–10) which is lower than current recommendations. These patients could have been understaged. Fifth, while a majority of the patients had solid PanNETs, a small number of patients had cystic PanNETs. Whether cystic features impact the performance of this model could not be assessed given the low sample size. As larger datasets become available, we will be able to address this in the future. Lastly, though we developed a model for preoperative prediction of nodal disease, we utilized size obtained at surgical pathology to develop the clinical–radiomics model. Though this is a postoperative variable, size on surgical pathology was found to be concordant with size on preoperative CT imaging in our study. Prior studies have also demonstrated concordance between radiologic (including CT imaging) and pathological size in PanNETs.58,59

In conclusion, this study presents a novel tool on the basis of RFs and clinical factors for accurate prediction of nodal disease in NF-PanNETs. Furthermore, we demonstrated that addition of RFs to clinical factors can make this model more robust and accurate. Further validation of this model is required to assess its performance in external cohorts. If validated, this tool could allow for non-invasive serial assessment of nodal disease in patients with well differentiated nonfunctioning PanNETs to tailor management plans and provide precise therapy on the basis of each patient’s disease biology.