Development and external validation of a machine learning model for predicting overall survival in head and neck adenoid cystic carcinoma based on the SEER database and a Chinese clinical cohort
Original Article

Development and external validation of a machine learning model for predicting overall survival in head and neck adenoid cystic carcinoma based on the SEER database and a Chinese clinical cohort

Yuanfeng Jiang1,2 ORCID logo, Xiansu Zhang2, Haofeng Qiu1,2, Peng Zhang2

1School of Medicine, University of Electronic Science and Technology of China, Chengdu, China; 2Department of Radiation, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, Chengdu, China

Contributions: (I) Conception and design: Y Jiang, P Zhang; (II) Administrative support: P Zhang; (III) Provision of study materials or patients: X Zhang, P Zhang; (IV) Collection and assembly of data: Y Jiang, H Qiu, X Zhang; (V) Data analysis and interpretation: Y Jiang, H Qiu; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

Correspondence to: Peng Zhang, MD, PhD. Department of Radiation, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, University of Electronic Science and Technology of China, No. 55, Section 4, South Renmin Road, Chengdu 610041, China. Email: izhangpeng@163.com.

Background: Adenoid cystic carcinoma (ACC) of the head and neck is a rare malignancy with high rates of perineural invasion (PNI) and distant metastasis. Although the tumor-node-metastasis (TNM) staging system provides a crucial anatomical foundation, it may not fully capture the biological heterogeneity of ACC. Existing predictive models often lack robust external validation or contain too many variables for routine clinical use. This study aimed to develop and validate a simplified prognostic model using a multi-algorithm machine learning (ML) consensus approach.

Methods: This retrospective study included patients with pathologically confirmed primary head and neck ACC and complete follow-up data from two cohorts: a training cohort from the Surveillance, Epidemiology, and End Results (SEER) database (n=2,870) and an external validation cohort from Sichuan Cancer Hospital (n=172). The primary endpoint was overall survival (OS), defined as a binary outcome (all-cause mortality vs. survival). Clinical variables were extracted from SEER via database queries and from the external cohort via medical records and telephone interviews. The SEER cohort was divided into training and internal testing sets at a 7:3 ratio. Traditional Cox regression was first applied to establish a prognostic baseline and construct a clinical nomogram. Subsequently, four ML algorithms—logistic regression (LR), classification and regression tree (CART), random forest (RF), and support vector machine (SVM)—were used to build prognostic models. A SHapley Additive exPlanations (SHAP)-based consensus strategy was then applied across the four algorithms specifically for objective feature selection to identify a minimal set of core prognostic features.

Results: A total of 3,042 patients were analyzed. In the external cohort, the median follow-up was 86.0 months, with 49 mortality events (28.5%) observed. Six core prognostic features were identified: M stage, age, brain metastasis, T stage, surgery, and N stage. The simplified LR model achieved the best performance among all tested algorithms, with an area under the curve (AUC) of 0.825 in the internal testing set. In the external validation cohort, it achieved an AUC of 0.817, outperforming the other three algorithms, with good calibration observed.

Conclusions: This study developed a simplified prognostic model for head and neck ACC based on six core features. The model showed promising discrimination in both the internal testing set (AUC =0.825) and the external validation cohort (AUC =0.817). However, the external cohort was relatively small (n=172), which limits our ability to fully assess the model’s generalizability. These findings should therefore be interpreted with caution. Future prospective studies with larger, multi-center cohorts are needed to confirm these results before the model can be recommended for routine clinical use.

Keywords: Carcinoma; adenoid cystic; head and neck neoplasms; machine learning (ML); Surveillance, Epidemiology, and End Results program (SEER program); survival analysis


Submitted Feb 11, 2026. Accepted for publication Apr 15, 2026. Published online Jun 26, 2026.

doi: 10.21037/gs-2026-1-0107


Highlight box

Key findings

• This study developed and externally validated a simplified machine learning (ML) prognostic model for head and neck adenoid cystic carcinoma (ACC) using six core features (M stage, age, brain metastasis, T stage, surgery, and N stage). The simplified logistic regression model demonstrated consistent discrimination in both the internal testing set [area under the curve (AUC) =0.825] and an independent Chinese clinical validation cohort (AUC =0.817).

What is known and what is new?

• While the tumor-node-metastasis staging system provides a crucial anatomical foundation, it may not fully capture the inherent biological heterogeneity of ACC. Existing prognostic models often retain an excessive number of variables and lack rigorous independent external validation.

• This study implemented a two-stage analytical framework. Traditional Cox regression was first applied to establish a prognostic baseline and construct a clinical nomogram. Building upon this foundation, a multi-algorithm ML consensus and SHapley Additive exPlanations interpretability were subsequently employed for objective feature selection, streamlining the model without compromising predictive performance. Additionally, distinct prognostic weights were identified for specific metastatic sites, suggesting that bone and brain metastases may carry a higher mortality risk than lung metastasis.

What is the implication, and what should change now?

• The externally validated simplified model may serve as a practical adjunctive tool for survival risk stratification in clinical practice. For patients classified as high-risk or those with advanced T4 disease, targeted screening such as brain magnetic resonance imaging and bone scintigraphy may be considered alongside routine chest imaging.


Introduction

Adenoid cystic carcinoma (ACC) of the head and neck is a rare malignancy, which primarily originates from the major and minor salivary glands. However, it also arises at other anatomical sites, including the nasal cavity, paranasal sinuses, larynx, and trachea (1,2). Clinically, ACC shows a unique biological behavior: although it typically exhibits an indolent course with a slow growth rate, it also demonstrates a high tendency for perineural invasion (PNI) and distant metastasis, particularly to the lungs, which leads to poor long-term outcomes (3,4). Consequently, while the 5-year overall survival (OS) rate is generally favorable, survival rates show a continuous decline over 10 to 15 years (5-7). This persistent attrition reflects the profound clinical and biological heterogeneity inherent to ACC. Consequently, such an unpredictable disease trajectory, frequently characterized by late-onset metastasis, is widely recognized as a major challenge for accurate long-term risk assessment (8).

Currently, the American Joint Committee on Cancer (AJCC) tumor-node-metastasis (TNM) staging system is the primary clinical standard for evaluating the prognosis of head and neck ACC. However, it is widely acknowledged in the field that because this system is primarily based on macroscopic anatomical information, its prognostic accuracy may be constrained by the profound biological heterogeneity of ACC. To provide a more comprehensive assessment, researchers have developed multidimensional prognostic models. For example, studies by Mu et al. (9), Pan et al. (10), and Chen et al. (11) constructed clinical nomograms to predict survival outcomes based on traditional regression methods. These existing models effectively summarize currently known prognostic factors, such as patient age, tumor site, PNI, surgical modality, and TNM stage (12-14), demonstrating acceptable to robust predictive accuracies with reported concordance indices and area under the curve (AUC) values generally ranging from 0.70 to over 0.80.

Despite the prognostic insights provided by traditional regression models, important challenges remain in their clinical translation. While ML algorithms offer advantages in capturing complex, non-linear variable interactions, comparative studies have shown that they do not consistently outperform traditional Cox models in OS prediction for cancer patients (15). Therefore, the primary challenge in this field lies not in maximizing AUC alone, but in improving model simplicity and generalizability. On one hand, traditional stepwise regression often retains an excessive number of variables, reducing the practicality of the resulting nomograms in routine clinical settings. On the other hand, the majority of newly developed models lack rigorous independent external validation, leaving their real-world applicability uncertain (16-18).

To address these limitations, this study implemented a two-stage analytical framework. Rather than replacing conventional statistics, we first applied Cox regression to establish a reliable prognostic baseline. We then employed a consensus strategy across four ML algorithms coupled with SHapley Additive exPlanations (SHAP)-based interpretability analysis to objectively identify a minimal set of core prognostic features from the large-scale Surveillance, Epidemiology, and End Results (SEER) database. In this design, ML was used specifically for feature selection rather than to maximize predictive metrics. Using only this simplified feature set, we subsequently conducted external validation in an independent cohort of Chinese patients to evaluate the real-world performance of the resulting model. This approach aimed to develop a prognostic model for head and neck ACC that balances predictive performance with clinical feasibility and cross-population applicability. We present this article in accordance with the TRIPOD reporting checklist (available at https://gs.amegroups.com/article/view/10.21037/gs-2026-1-0107/rc).


Methods

Data source and study population

This retrospective study included two independent cohorts. The training cohort data were obtained from the SEER database of the National Cancer Institute (NCI), covering patients diagnosed with head and neck ACC between 2004 and 2019. The external validation cohort included patients with head and neck ACC treated at Sichuan Cancer Hospital & Institute between January 2010 and December 2019.

Both cohorts were subject to identical inclusion and exclusion criteria. Inclusion criteria were: (I) pathologically confirmed ACC; (II) primary tumor located in the head and neck region; and (III) ACC as the first or only primary malignancy. Exclusion criteria were: (I) age <18 years; (II) survival time recorded as 0 months or unknown; (III) missing key clinical variables; and (IV) diagnosis based solely on death certificate or autopsy.

A complete-case analysis approach was adopted for handling missing data. For the SEER cohort, complete records were directly extracted. For the external validation cohort, 13 out of 185 initially eligible patients (a missing rate of approximately 7.0%) were excluded due to missing key clinical or pathological variables. Given the low proportion of missing data (<10%), no data imputation techniques were applied to avoid introducing artificial bias. Following the application of inclusion and exclusion criteria, a post-hoc sample size assessment was conducted for the external validation cohort. Sample adequacy was evaluated using two complementary approaches: the classical events per variable (EPV) rule and the Riley criteria for binary outcome prediction models, implemented via the pmsampsize R package. For the Riley calculation, a conservative expected AUC =0.75, the number of predictor parameters, and the observed outcome prevalence in the external cohort were specified as inputs. It should be noted that EPV was originally proposed in the context of model development and is applied here for reference only.

As the SEER database is publicly available and de-identified, ethical review was waived for the training cohort. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study protocol for the external validation cohort was approved by the Ethics Committee of Sichuan Cancer Hospital, and the requirement for informed consent was waived due to the retrospective nature of the study.

Variable selection and endpoints

Demographic, pathological, and treatment data were extracted for all eligible patients using standardized protocols. For the SEER cohort, data were obtained via SEER*Stat software. For the external validation cohort, clinical data were systematically extracted from electronic medical records by trained clinical investigators. The extracted variables included age at diagnosis, gender, race, marital status, primary tumor site, TNM stage, and treatment modalities (surgery, radiotherapy, chemotherapy, and neck dissection). To ensure clinical consistency across the long temporal span of both cohorts, tumor staging was based on the AJCC Cancer Staging Manual edition applicable at the time of diagnosis.

The primary endpoint was OS, defined as the time from the date of initial diagnosis to the date of death from any cause or the date of last follow-up. For the Sichuan Cancer Hospital cohort, follow-up was conducted through outpatient review records and telephone interviews. The median follow-up duration was 69.0 months for the SEER training cohort and 86.0 months for the external validation cohort, covering the 5 to 10-year observation window considered appropriate for capturing late survival events in head and neck ACC.

Statistical analysis

Data analysis was performed using R software (version 4.5.2). For baseline characteristics, categorical variables were compared using the Chi-squared test. Survival curves were generated using the Kaplan-Meier method and compared using the log-rank test.

To identify independent prognostic factors, we first conducted a univariate Cox regression analysis in the SEER cohort. Variables with a P value <0.10 were entered into the multivariate Cox proportional hazards model. Variables with a P value <0.05 in the multivariate analysis were considered statistically significant. Based on these independent prognostic factors, a nomogram was constructed to predict 3-, 5-, and 8-year OS rates. The discriminatory ability of the model was evaluated using the receiver operating characteristic (ROC) curve and the AUC. Based on established conventions in oncology prognostic modeling, an AUC of 0.70–0.75 was defined as indicating acceptable discrimination, and an AUC ≥0.75 was considered to indicate good discrimination and potential clinical utility. Additionally, patients were stratified into high- and low-risk groups based on the median risk score calculated from the model, and survival differences were analyzed.

ML and feature selection

The SEER cohort was randomly divided into a training set and an internal testing set at a 7:3 ratio. To ensure a balanced distribution of the target endpoints, this split was conducted using a stratified random sampling approach based on the binary survival status, maintaining a consistent outcome prevalence across both sets. The independent Sichuan Cancer Hospital cohort served directly as the external validation set without further splitting. For the ML models, the target variable was defined as the binary OS status (dead vs. alive) over the entire follow-up period. We employed four ML algorithms—logistic regression (LR), classification and regression tree (CART), random forest (RF), and support vector machine (SVM)—to construct prognostic models using all available variables. Given our sufficiently long median follow-up time, this binary classification strategy effectively captures the long-term prognostic signals. Model performance was evaluated using the AUC and Brier score.

To identify the most robust prognostic features, we used the SHAP method to interpret the importance of features across the different models. A consensus strategy was applied by using a Venn diagram to intersect the top-ranked features from the four models, thereby identifying a set of core prognostic features. Finally, the four models were retrained using only this simplified core feature set. The performance and generalization capability of the simplified models were validated in the independent external cohort by calculating the AUC.


Results

Baseline characteristics

This study evaluated 3,042 patients with head and neck ACC. The training cohort (SEER database) consisted of 2,870 patients, while the external validation cohort (our independent hospital cohort) comprised 172 patients; for our entire cohort, the median age at diagnosis was 58 years. The majority of patients were female (1,760 cases, 57.9%), and the most common primary site was the major salivary glands (1,499 cases, 49.3%).

Comparisons between the training and validation cohorts revealed statistically significant differences in age distribution, marital status, primary site, T stage, and N stage (all P<0.05) (Table 1). Notably, compared with the training set, patients in the validation set had more advanced clinical manifestations. The proportion of patients with T4 stage tumors was significantly higher in the validation set than in the training set (48.8% vs. 27.0%), as was the rate of positive lymph node metastasis (N+: 39.0% vs. 12.7%). Conversely, the proportion of elderly patients (≥75 years) was lower in the validation set (2.3% vs. 16.0%). No significant differences were observed between the two groups in gender distribution or M stage (P>0.05).

Table 1

Comparison of clinicopathological characteristics between patients in the training set and the validation set

Clinical characteristics Training cohort (n=2,870) Validation cohort (n=172) Total (n=3,042) P value
Marital status <0.001
   Married 1,654 (57.6) 150 (87.2) 1,804 (59.3)
   Unmarried/other 1,216 (42.4) 22 (12.8) 1,238 (40.7)
Gender 0.59
   Female 1,667 (58.1) 93 (54.1) 1,760 (57.9)
   Male 1,203 (41.9) 79 (45.9) 1,282 (42.1)
Age (years) <0.001
   0–44 597 (20.8) 64 (37.2) 661 (21.7)
   45–59 932 (32.5) 60 (34.9) 992 (32.6)
   60–74 882 (30.7) 44 (25.6) 926 (30.4)
   ≥75 459 (16.0) 4 (2.3) 463 (15.2)
Primary site 0.043
   Oral cavity 700 (24.4) 33 (19.2) 733 (24.1)
   Pharynx 186 (6.5) 9 (5.2) 195 (6.4)
   Larynx 53 (1.8) 2 (1.2) 55 (1.8)
   Nasal cavity & sinuses 509 (17.7) 51 (29.7) 560 (18.4)
   Salivary glands 1,422 (49.5) 77 (44.8) 1,499 (49.3)
T stage <0.001
   T0 8 (0.3) 0 (0) 8 (0.3)
   T1 687 (23.9) 10 (5.8) 697 (22.9)
   T2 587 (20.5) 43 (25.0) 630 (20.7)
   T3 470 (16.4) 26 (15.1) 496 (16.3)
   T4 774 (27.0) 84 (48.8) 858 (28.2)
   Tx 344 (12.0) 9 (5.2) 353 (11.6)
N stage <0.001
   N0 2,234 (77.8) 93 (54.1) 2,327 (76.5)
   N1 184 (6.4) 33 (19.2) 217 (7.1)
   N2 162 (5.6) 33 (19.2) 195 (6.4)
   N3 21 (0.7) 1 (0.6) 22 (0.7)
   Nx 269 (9.4) 12 (7.0) 281 (9.2)
M stage 0.17
   M0 2,450 (85.4) 152 (88.4) 2,602 (85.5)
   M1 236 (8.2) 17 (9.9) 253 (8.3)
   Mx 184 (6.4) 3 (1.7) 187 (6.1)
Race <0.001
   Black 305 (10.6) 0 (0.0) 305 (10.0)
   Other/unknown 426 (14.8) 172 (100.0) 598 (19.7)
   White 2,139 (74.5) 0 (0.0) 2,139 (70.3)
Vital status <0.001
   Alive 1,672 (58.3) 123 (71.5) 1,795 (59.0)
   Dead 1,198 (41.7) 49 (28.5) 1,247 (41.0)
Follow-up time (months) 69.0 [34, 119] 86.0 [65, 104] 71.0 [35, 117] 0.07

Data are presented as n (%) or median [IQR]. IQR, interquartile range; M, metastasis; N, node; T, tumor.

Prognostic factors and nomogram construction

Univariate Cox regression analysis identified variables with a P<0.10, which were included in multivariate analysis. The multivariate Cox proportional hazards analysis (Figure 1) revealed several independent prognostic factors.

Figure 1 Forest plot of multivariate Cox regression analysis for independent prognostic factors in head and neck ACC. ACC, adenoid cystic carcinoma; CI, confidence interval; coef, coefficient; HR, hazard ratio; M, metastasis; N, node; T, tumor.

In terms of demographic characteristics, advanced age was a significant risk factor for poor prognosis. Compared to patients aged ≤44 years, those aged 60–74 years [hazard ratio (HR) =2.15, P<0.001] and ≥75 years (HR =4.09, P<0.001) had a significantly higher risk of mortality. Gender (HR =1.18, P=0.006) and unmarried status (HR =1.21, P=0.002) were also associated with poorer survival.

Regarding clinicopathological features, patients with primary tumors in the nasal cavity and paranasal sinuses had a significantly higher risk of death compared to those with oral cavity tumors (HR =1.59, P<0.001). Advanced T stage (T3, T4) and lymph node metastasis (N1–N3) showed a clear gradient effect on risk (all P<0.05). Distant metastasis (M1) substantially increased the risk of mortality (HR =1.93, P<0.001). Subgroup analysis indicated that bone metastasis (HR =1.46), brain metastasis (HR =3.86), and liver metastasis (HR =2.69) were independent risk factors for poor prognosis (all P<0.05), whereas lung metastasis did not show statistical significance in this model (P>0.05).

Regarding treatment, patients who did not undergo surgery had a 1.92-fold higher risk of death compared to those who received radical surgery (P<0.001). In this study, race, radiotherapy alone, chemotherapy alone, and neck dissection were not independently associated with OS (P>0.05).

Based on the seven identified independent prognostic factors (age, marital status, primary site, T stage, N stage, M stage, and surgery), a nomogram was constructed to predict 3-, 5-, and 8-year OS (Figure 2). Using the median risk score (150.97) as the cutoff, patients were stratified into low-risk (≤150.97) and high-risk (>150.97) groups. Kaplan-Meier analysis indicated a statistically significant difference in survival curves between the two groups (P<0.001, Figure 3). The 3-, 5-, and 8-year OS rates for the low-risk group were 94.6%, 90.3%, and 83.0%, respectively, which were significantly superior to the 68.1%, 54.7%, and 38.6% observed in the high-risk group.

Figure 2 Nomogram for predicting 3-, 5-, and 8-year OS in patients with head and neck ACC. To use the nomogram, the specific points of each variable are identified on the top scale and summed. The total points projected to the bottom scale indicate the probabilities of 3-, 5-, and 8-year OS. ACC, adenoid cystic carcinoma; M, metastasis; N, node; OS, overall survival; T, tumor.
Figure 3 Kaplan-Meier survival curves based on nomogram risk stratification.

ML models and feature importance analysis

In the SEER internal test set, all four ML algorithms (LR, CART, RF, and SVM) exhibited predictive performances superior to chance. Among them, the LR model (AUC =0.796) and the RF model (AUC =0.792) showed the best discrimination, followed by the SVM model (AUC =0.782). The CART model exhibited relatively limited predictive utility (AUC =0.764) (Figure 4A). Regarding calibration, the LR model achieved the lowest Brier score (0.184), indicating that its predicted probabilities most closely aligned with the actual survival outcomes (Figure 4B).

Figure 4 Evaluation of predictive performance of ML models in the internal testing set. (A) ROC curves of models in the internal testing set; (B) evaluation of calibration (Brier scores) of models in the internal testing set. AUC, area under the curve; CART, classification and regression tree; LR, logistic regression; ML, machine learning; RF, random forest; ROC, receiver operating characteristic; SVM, support vector machine.

SHAP interpretability analysis (Figure 5) revealed both similarities and differences in the decision-making logic of the different algorithms. Age, TNM stage, and distant metastasis were identified as core prognostic features across all models. However, the models weighed specific metastatic sites differently. The SVM and LR models assigned extremely high weights to bone metastasis and brain metastasis, with their importance scores exceeding that of the overall M stage itself. In contrast, the RF and CART models tended to rely more on the overall status of age and M stage.

Figure 5 Model interpretability analysis based on the SHAP method—global feature importance ranking (mean SHAP values). (A) LR; (B) RF; (C) SVM; (D) CART. CART, classification and regression tree; LR, logistic regression; M, metastasis; N, node; RF, random forest; SHAP, SHapley Additive exPlanations; SVM, support vector machine; T, tumor.

The SHAP summary plot (Figure 6) provided additional information about the direction of feature impact: Data points representing advanced age (≥ 75 years), late T/N stages, and the presence of bone, brain, or liver metastasis were mainly distributed in the positive SHAP value region, indicating that these features significantly increase the risk of death.

Figure 6 Model interpretability analysis based on the SHAP method—SHAP summary plots illustrating the direction of feature impact on prognosis. (A) LR; (B) RF; (C) SVM; (D) CART. CART, classification and regression tree; LR, logistic regression; M, metastasis; N, node; RF, random forest; SHAP, SHapley Additive exPlanations; SVM, support vector machine; T, tumor.

Construction and validation of the simplified model

To construct a streamlined model highly feasible for routine clinical practice, we identified the top 10 most important features from each of the four ML models. Using a Venn diagram to find the intersection, we identified six core prognostic features: M stage, age, brain metastasis, T stage, surgery, and N stage (Figure 7A).

Figure 7 Feature selection based on ML and performance evaluation in external validation. (A) Venn diagram of feature selection by four algorithms; (B) ROC curves of simplified models in the external validation set. AUC, area under the curve; CART, classification and regression tree; LR, logistic regression; ML, machine learning; RF, random forest; ROC, receiver operating characteristic; SVM, support vector machine.

Subsequently, we retrained the four models using exclusively this core feature set and evaluated their performance in an independent external cohort (n=172) (Figure 7B). The results indicated that the simplified LR model had the highest performance on the external validation set, with an AUC value of 0.817. This was significantly superior to the RF (AUC =0.775), SVM (AUC =0.719), and CART (AUC =0.683) models. Furthermore, when comparing the performance between the internal testing set and the external validation set (Figure 8), the LR model showed the smallest difference in AUC (internal 0.825 vs. external 0.817). These results suggest that the model may generalize reasonably well across diverse patient populations.

Figure 8 Comparison of AUC values of models between the SEER internal testing set and external validation set. AUC, area under the curve; CART, classification and regression tree; LR, logistic regression; RF, random forest; SEER, Surveillance, Epidemiology, and End Results; SVM, support vector machine.

Discussion

This study developed and validated a prognostic model for head and neck ACC utilizing a large-scale population from the SEER database. Our analytical framework integrated traditional survival analyses with ML algorithms to balance clinical interpretability and feature optimization. Specifically, we performed univariate and multivariate Cox regression analyses to identify independent prognostic factors, construct a clinical nomogram, and stratify survival risk via Kaplan-Meier curves. Building upon this traditional prognostic foundation, we utilized multiple ML algorithms and SHAP value interpretations for robust feature selection. This consensus strategy enabled the identification of six core prognostic factors: M stage, age, brain metastasis, T stage, surgery, and N stage, thereby streamlining the model to reduce clinical workload without compromising predictive performance. Based on these core features, a simplified LR model demonstrated an AUC of 0.825 in the SEER training set and maintained an AUC of 0.817 alongside an acceptable Brier score in the independent external validation cohort. Because both AUC values exceed our pre-specified threshold (AUC ≥0.75), these results indicate that the model achieves clinically meaningful discrimination, suggesting its potential utility for long-term survival risk stratification in patients with head and neck ACC.

The core features identified in this study confirm that for resectable lesions, radical surgery is the key factor for improving OS, which is consistent with existing clinical consensus (19-21). Surgical treatment showed a clear advantage over radical radiotherapy alone. Radical radiotherapy is typically reserved for cases where the tumor is unresectable, or the patient is in poor physical condition, and its outcomes are generally inferior to surgery (22-24). A topic worth discussing is the value of postoperative radiotherapy (PORT). High-risk patients, such as those with positive margins, advanced T stages, or PNI, have PORT as the standard of care to reduce the risk of local recurrence (14,25-28). Studies by Chen et al. (29) and Gao et al. (30) demonstrated that a radiation dose of at least 60 Gy is required for effective local control, and doses up to 66 Gy may be needed for positive margins (23). However, this increase in local control does not necessarily translate to a significant improvement in OS (7,31,32). This finding supports the view of Cantu et al. (24), suggesting that local control does not directly lead to long-term survival benefits, because distant metastasis is often the most important factor for the patient’s final outcome (22).

Regarding systemic therapy, because of the insufficient prognostic weight of chemotherapy, this factor was not included in the final model. It matches the biological feature of head and neck ACC being rather unresponsive to conventional cytotoxic medicines. Previous studies have shown that chemotherapy is mainly used for palliative care or as a radiosensitizer, but it offers limited survival benefit after adjusting for disease stage and surgery (3,4,33-35). Our study also found that while N stage is a core prognostic factor, neck dissection (the surgical procedure itself) was not an independent protective factor after adjusting for staging. This is consistent with meta-analyses by Xiao et al. (36) and Suton et al. (37), which suggest that for clinically node-negative patients, prophylactic neck dissection may not directly extend OS. This suggests that the prognosis is more related to how much the tumor has spread biologically (as seen in the N stage) rather than to the preventive removal of lymph nodes. However, this conclusion should be applied with caution depending on the primary site. Since Amit et al. (38) pointed out that oral ACC has a high rate of occult metastasis (up to 37%), elective neck dissection remains clinically important for oral ACC to ensure accurate staging and guide subsequent treatment (39).

One finding of interest in this study is the insight into metastasis patterns provided by ML interpretability (SHAP). While lung metastasis is the most common form of distant spread, our model assigned higher risk weights to brain and bone metastases. This statistical signal is consistent with clinical observations, where ACC lung metastases typically exhibit an indolent biological behavior. In clinical practice, it is not uncommon for patients to present with minimal or subtle symptoms despite the presence of radiologically confirmed multiple pulmonary lesions. Consequently, active surveillance is recommended as the initial management strategy for patients with asymptomatic lung metastases. Subsequently, if the disease is oligometastatic and amenable to local intervention, local ablative therapies such as pulmonary metastasectomy or stereotactic body radiotherapy (SBRT) may be considered to achieve disease control and delay the need for systemic therapy (40). However, current evidence suggests that local treatment of metastatic lesions does not confer a significant OS or progression-free survival advantage compared to observation alone in unselected recurrent or metastatic ACC populations (41,42). The potential benefit of local ablative therapies may be limited to carefully selected patients, such as those with oligometastatic disease and small-volume pulmonary involvement, though this remains to be confirmed in prospective studies. In contrast, bone and brain metastases are associated with more aggressive biological behavior and rapid clinical deterioration. Therefore, a risk-stratified surveillance approach may be warranted. For patients classified as high-risk by our model or those presenting with advanced T4 disease, brain magnetic resonance imaging (MRI) and bone scintigraphy should be considered as part of a tailored follow-up strategy, alongside routine physical examinations and chest imaging. However, we must acknowledge that the limited number of site-specific metastases in our cohort precludes reliable subgroup analyses. These observations are proposed as hypothesis-generating considerations, and prospective validation in larger cohorts is required to establish definitive screening protocols.

Several broader limitations of this study must also be acknowledged. First, the sample size of our external validation cohort is a recognized limitation. While the classical EPV ratio was 8.2, we acknowledge that our cohort does not fully meet the stringent contemporary Riley criteria for binary prediction models. This constraint is primarily attributable to the rarity of ACC and the inherent difficulty of assembling a large independent validation cohort within a single center. Second, there is a baseline imbalance between the cohorts, with the Chinese validation set containing a higher proportion of advanced-stage (T4 and N+) patients. This discrepancy largely reflects the distinct patient populations between a population-based registry (SEER), which captures a broad distribution of early-stage cases, and a regional tertiary cancer center, which selectively receives complex cases requiring multidisciplinary interventions. For instance, studies by Wu et al. (43) have observed that primary ACC in Chinese cohorts frequently presents at an advanced clinical T-stage at the time of initial diagnosis. Similarly, research by Chen et al. (11) evaluating head and neck ACC prognostic models also documented baseline population and staging disparities between SEER training data and Chinese institutional validation cohorts. Consequently, as our validation cohort is enriched with advanced disease, the model’s generalizability across the full clinical spectrum remains to be confirmed, especially regarding early-stage T1/T2 cases. Third, due to the inherent limitations of the SEER registry, our model lacks microscopic pathological variables such as PNI and surgical margin status. In multidisciplinary practice, these features are important prognostic indicators relied upon by clinicians to tailor adjuvant treatments. The omission of these variables constrains our model’s capacity to provide highly individualized surgical or radiotherapeutic guidance. Finally, the retrospective design of this study introduces inherent limitations, including potential selection bias and information bias, which may affect the generalizability of our findings. Future prospective, multicenter studies with larger cohorts and comprehensive pathological data are warranted to further validate this model.


Conclusions

In summary, this study integrated traditional Cox regression with a multi-algorithm ML consensus strategy to identify six core prognostic features: M stage, age, brain metastasis, T stage, surgery, and N stage. The resulting simplified model demonstrated promising discrimination in both internal testing and external validation, with AUC values exceeding the pre-specified threshold of 0.75. Given its reduced variable requirement, the model is potentially feasible for routine clinical use. We propose that it may serve as a practical adjunctive tool to assist clinicians in risk stratification and personalized management of head and neck ACC, pending further validation in larger prospective cohorts.


Acknowledgments

None.


Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-1-0107/rc

Data Sharing Statement: Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-1-0107/dss

Peer Review File: Available at https://gs.amegroups.com/article/view/10.21037/gs-2026-1-0107/prf

Funding: This study was supported by the Beijing Bethune Charitable Foundation through the 2024 Colorectal Cancer and Head and Neck Tumor Innovation Incubation Research Fund (No. 2024-YJ-102-J).

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-2026-1-0107/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. The study was approved by the Ethics Committee for Medical Research and New Medical Technology of Sichuan Cancer Hospital, and the requirement for informed consent was waived due to the retrospective nature of the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.


References

  1. Coca-Pelaz A, Rodrigo JP, Bradley PJ, et al. Adenoid cystic carcinoma of the head and neck--An update. Oral Oncol 2015;51:652-61. [Crossref] [PubMed]
  2. Laurie SA, Ho AL, Fury MG, et al. Systemic therapy in the management of metastatic or locally recurrent adenoid cystic carcinoma of the salivary glands: a systematic review. Lancet Oncol 2011;12:815-24. [Crossref] [PubMed]
  3. Zhou J, Zhao G, Wang S, et al. Systemic therapy in the management of metastatic or locally recurrent adenoid cystic carcinoma of the salivary glands: a systematic review of the last decade. Br J Cancer 2024;131:1021-31. [Crossref] [PubMed]
  4. Jia Y, Liu Y, Yang H, et al. Adenoid cystic carcinoma: insights from molecular characterization and therapeutic advances. MedComm (2020) 2024;5:e734.
  5. Ouyang DQ, Liang LZ, Zheng GS, et al. Risk factors and prognosis for salivary gland adenoid cystic carcinoma in southern china: A 25-year retrospective study. Medicine (Baltimore) 2017;96:e5964. [Crossref] [PubMed]
  6. Ullah A, Ahmed A, Lee KT, et al. Salivary gland adenoid cystic carcinoma in the U.S population: Importance of grade, site of metastases, and adjuvant radiation for survival. Am J Otolaryngol 2024;45:104266. [Crossref] [PubMed]
  7. Zupancic M, Näsman A, Berglund A, et al. Adenoid Cystic Carcinoma (AdCC): A Clinical Survey of a Large Patient Cohort. Cancers (Basel) 2023;15:1499. [Crossref] [PubMed]
  8. Lukšić I, Tarle M, Raguž M, et al. Persistent Long-Term Risk After Primary Surgery for Head and Neck Adenoid Cystic Carcinoma: Competing-Risk and Conditional Estimates. Cancers (Basel) 2026;18:833. [Crossref] [PubMed]
  9. Mu X, Li Y, He L, et al. Prognostic nomogram for adenoid cystic carcinoma in different anatomic sites. Head Neck 2021;43:48-59. [Crossref] [PubMed]
  10. Pan Y, Zhang L, Xu S, et al. Development and validation of a nomogram for predicting overall survival of head and neck adenoid cystic carcinoma. Sci Rep 2024;14:26406. [Crossref] [PubMed]
  11. Chen X, Liu R, Wu Y, et al. A competing risk-based prognostic model for cancer-specific survival in non-metastatic head and neck adenoid cystic carcinoma. Front Oncol 2026;16:1752964. [Crossref] [PubMed]
  12. de Morais EF, da Silva LP, Moreira DGL, et al. Prognostic Factors and Survival in Adenoid Cystic Carcinoma of the Head and Neck: A Retrospective Clinical and Histopathological Analysis of Patients Seen at a Cancer Center. Head Neck Pathol 2021;15:416-24. [Crossref] [PubMed]
  13. Atallah S, Casiraghi O, Fakhry N, et al. A prospective multicentre REFCOR study of 470 cases of head and neck Adenoid cystic carcinoma: epidemiology and prognostic factors. Eur J Cancer 2020;130:241-9. [Crossref] [PubMed]
  14. Tasoulas J, Divaris K, Theocharis S, et al. Impact of Tumor Site and Adjuvant Radiotherapy on Survival of Patients with Adenoid Cystic Carcinoma: A SEER Database Analysis. Cancers (Basel) 2021;13:589. [Crossref] [PubMed]
  15. Huang Y, Bazzazzadehgan S, Li J, et al. Comparison of machine learning methods versus traditional Cox regression for survival prediction in cancer using real-world data: a systematic literature review and meta-analysis. BMC Med Res Methodol 2025;25:243. [Crossref] [PubMed]
  16. Balachandran VP, Gonen M, Smith JJ, et al. Nomograms in oncology: more than meets the eye. Lancet Oncol 2015;16:e173-80. [Crossref] [PubMed]
  17. Peng Z, Wang Y, Wang Y, et al. Application of radiomics and machine learning in head and neck cancers. Int J Biol Sci 2021;17:475-86. [Crossref] [PubMed]
  18. Xu XL, Xu JH, He JQ, et al. Novel prognostic nomograms for postoperative patients with oral cavity squamous cell carcinoma in the central region of China. BMC Cancer 2024;24:730. [Crossref] [PubMed]
  19. Ko JJ, Siever JE, Hao D, et al. Adenoid cystic carcinoma of head and neck: clinical predictors of outcome from a Canadian centre. Curr Oncol 2016;23:26-33. [Crossref] [PubMed]
  20. Mendenhall WM, Morris CG, Amdur RJ, et al. Radiotherapy alone or combined with surgery for adenoid cystic carcinoma of the head and neck. Head Neck 2004;26:154-62. [Crossref] [PubMed]
  21. Cheng Y, Xu L, Chen Z, et al. Prognosis of adenoid cystic carcinoma in head and neck region treated with different regimens-A single-centre study. Cancer Med 2023;12:2368-77. [Crossref] [PubMed]
  22. Chen AM, Bucci MK, Weinberg V, et al. Adenoid cystic carcinoma of the head and neck treated by surgery with or without postoperative radiation therapy: prognostic features of recurrence. Int J Radiat Oncol Biol Phys 2006;66:152-9. [Crossref] [PubMed]
  23. Garden AS, Weber RS, Morrison WH, et al. The influence of positive margins and nerve invasion in adenoid cystic carcinoma of the head and neck treated with surgery and radiation. Int J Radiat Oncol Biol Phys 1995;32:619-26. [Crossref] [PubMed]
  24. Cantù G. Adenoid cystic carcinoma. An indolent but aggressive tumour. Part B: treatment and prognosis. Acta Otorhinolaryngol Ital 2021;41:296-307.
  25. Lee A, Givi B, Osborn VW, et al. Patterns of care and survival of adjuvant radiation for major salivary adenoid cystic carcinoma. Laryngoscope 2017;127:2057-62. [Crossref] [PubMed]
  26. Ali S, Palmer FL, Katabi N, et al. Long-term local control rates of patients with adenoid cystic carcinoma of the head and neck managed by surgery and postoperative radiation. Laryngoscope 2017;127:2265-9. [Crossref] [PubMed]
  27. Jensen AD, Poulakis M, Nikoghosyan AV, et al. High-LET radiotherapy for adenoid cystic carcinoma of the head and neck: 15 years' experience with raster-scanned carbon ion therapy. Radiother Oncol 2016;118:272-80. [Crossref] [PubMed]
  28. Barrett AW, Speight PM. Perineural invasion in adenoid cystic carcinoma of the salivary glands: a valid prognostic indicator? Oral Oncol 2009;45:936-40. [Crossref] [PubMed]
  29. Chen Y, Zheng ZQ, Chen FP, et al. Role of Postoperative Radiotherapy in Nonmetastatic Head and Neck Adenoid Cystic Carcinoma. J Natl Compr Canc Netw 2020;18:1476-84. [Crossref] [PubMed]
  30. Gao RW, Routman DM, Harmsen WS, et al. Adenoid cystic carcinoma of the head and neck: Patterns of recurrence and implications for intensity-modulated radiotherapy. Head Neck 2023;45:187-96. [Crossref] [PubMed]
  31. Choi SH, Yang AJ, Yoon SO, et al. Role of postoperative radiotherapy in resected adenoid cystic carcinoma of the head and neck. Radiat Oncol 2022;17:197. [Crossref] [PubMed]
  32. Bjørndal K, Krogdahl A, Therkildsen MH, et al. Salivary adenoid cystic carcinoma in Denmark 1990-2005: Outcome and independent prognostic factors including the benefit of radiotherapy. Results of the Danish Head and Neck Cancer Group (DAHANCA). Oral Oncol 2015;51:1138-42.
  33. Geiger JL, Ismaila N, Beadle B, et al. Management of Salivary Gland Malignancy: ASCO Guideline. J Clin Oncol 2021;39:1909-41. [Crossref] [PubMed]
  34. Sahara S, Herzog AE, Nör JE. Systemic therapies for salivary gland adenoid cystic carcinoma. Am J Cancer Res 2021;11:4092-110.
  35. Papaspyrou G, Hoch S, Rinaldo A, et al. Chemotherapy and targeted therapy in adenoid cystic carcinoma of the head and neck: a review. Head Neck 2011;33:905-11. [Crossref] [PubMed]
  36. Xiao R, Sethi RKV, Feng AL, et al. The role of elective neck dissection in patients with adenoid cystic carcinoma of the head and neck. Laryngoscope 2019;129:2094-104. [Crossref] [PubMed]
  37. Suton P, Luksic I. Prognostic value of elective neck dissection in adenoid cystic carcinoma of head and neck: a meta-analysis: A call for randomized trials and international consensus. Int J Oral Maxillofac Surg 2021;50:1403-7. [Crossref] [PubMed]
  38. Amit M, Binenbaum Y, Sharma K, et al. Incidence of cervical lymph node metastasis and its association with outcomes in patients with adenoid cystic carcinoma. An international collaborative study. Head Neck 2015;37:1032-7.
  39. Atallah S, Moya-Plana A, Malard O, et al. Should a neck dissection be performed on patients with cN0 adenoid cystic carcinoma? A REFCOR propensity score matching study. Eur J Cancer 2020;130:250-8.
  40. Lee RH, Wai KC, Chan JW, et al. Approaches to the Management of Metastatic Adenoid Cystic Carcinoma. Cancers (Basel) 2022;14:5698. [Crossref] [PubMed]
  41. Zhang D, Chai Y, Wei Y, et al. Survival and prognosis of metastatic head and neck adenoid cystic carcinoma. Head Neck 2022;44:2407-16. [Crossref] [PubMed]
  42. Xu MJ, Wu TJ, van Zante A, et al. Mortality risk after clinical management of recurrent and metastatic adenoid cystic carcinoma. J Otolaryngol Head Neck Surg 2018;47:28. [Crossref] [PubMed]
  43. Wu J, Cui H, Liang M, et al. Histological-pathological and clinical T stage of primary adenoid cystic carcinoma of the lacrimal gland in a Chinese population. BMC Cancer 2025;26:110. [Crossref] [PubMed]
Cite this article as: Jiang Y, Zhang X, Qiu H, Zhang P. Development and external validation of a machine learning model for predicting overall survival in head and neck adenoid cystic carcinoma based on the SEER database and a Chinese clinical cohort. Gland Surg 2026;15(6):169. doi: 10.21037/gs-2026-1-0107

Download Citation