Development and external validation of a machine learning model for predicting overall survival in head and neck adenoid cystic carcinoma based on the SEER database and a Chinese clinical cohort

Background: Adenoid cystic carcinoma (ACC) of the head and neck is a rare malignancy with high rates of perineural invasion (PNI) and distant metastasis. Although the tumor-node-metastasis (TNM) staging system provides a crucial anatomical foundation, it may not fully capture the biological heterogeneity of ACC. Existing predictive models often lack robust external validation or contain too many variables for routine clinical use. This study aimed to develop and validate a simplified prognostic model using a multi-algorithm machine learning (ML) consensus approach.

Methods: This retrospective study included patients with pathologically confirmed primary head and neck ACC and complete follow-up data from two cohorts: a training cohort from the Surveillance, Epidemiology, and End Results (SEER) database (n=2,870) and an external validation cohort from Sichuan Cancer Hospital (n=172). The primary endpoint was overall survival (OS), defined as a binary outcome (all-cause mortality vs. survival). Clinical variables were extracted from SEER via database queries and from the external cohort via medical records and telephone interviews. The SEER cohort was divided into training and internal testing sets at a 7:3 ratio. Traditional Cox regression was first applied to establish a prognostic baseline and construct a clinical nomogram. Subsequently, four ML algorithms—logistic regression (LR), classification and regression tree (CART), random forest (RF), and support vector machine (SVM)—were used to build prognostic models. A SHapley Additive exPlanations (SHAP)-based consensus strategy was then applied across the four algorithms specifically for objective feature selection to identify a minimal set of core prognostic features.

Results: A total of 3,042 patients were analyzed. In the external cohort, the median follow-up was 86.0 months, with 49 mortality events (28.5%) observed. Six core prognostic features were identified: M stage, age, brain metastasis, T stage, surgery, and N stage. The simplified LR model achieved the best performance among all tested algorithms, with an area under the curve (AUC) of 0.825 in the internal testing set. In the external validation cohort, it achieved an AUC of 0.817, outperforming the other three algorithms, with good calibration observed.

Conclusions: This study developed a simplified prognostic model for head and neck ACC based on six core features. The model showed promising discrimination in both the internal testing set (AUC =0.825) and the external validation cohort (AUC =0.817). However, the external cohort was relatively small (n=172), which limits our ability to fully assess the model’s generalizability. These findings should therefore be interpreted with caution. Future prospective studies with larger, multi-center cohorts are needed to confirm these results before the model can be recommended for routine clinical use.