A nomogram for intraoperatively predicting non-sentinel lymph node metastases in early breast cancer patients with positive sentinel lymph nodes

Introduction

Axillary lymph node status is a key prognostic factor affecting overall survival (OS) in breast cancer patients (1). In the past, 60% of breast patients staged N0 after routine axillary lymph node dissection (ALND), and several complications frequently occurred following ALND, including lymphedema, neuralgia, as well as limited movement in the shoulder and arm (2,3). Therefore, identifying patients who require ALND is necessary. Sentinel lymph node biopsy (SLNB), which was recommended by the National Comprehensive Cancer Network (NCCN), has been widely confirmed as a standard procedure for patients without positive SLNs instead of ALND (4,5).

Following the development of adjuvant therapy, the management of early breast cancer (cT1-2N0M0) entered a new era. The American College of Surgeons Oncology Group Z-0011 trial further changed the status of ALND. Early-stage patients with SLNB and standard adjuvant therapy, even if harboring a few involved sentinel lymph nodes (SLNs), could still obtain a favorable prognosis (6). Thus, patients with 1 to 2 SLN metastases who have undergone breast-conserving surgery and are scheduled to receive postoperative systemic therapy and radiation therapy would suffer less ALND and fewer complications. However, the necessity of complete ALND remains debatable in patients who receive total mastectomy and are willing to undergo radiotherapy or breast-conserving surgery with three or more SLN metastases. A tool to determine which patients cannot omit ALND may be needed, especially for patients who do not meet the Z-0011 trial criteria.

At present, only 50% of patients with positive SLNs have additional positive lymph nodes after complete ALND (2). These patients hardly benefit from complete ALND. Therefore, models have been developed to predict the risk of additional non-SLN metastases (7-9), among which the Memorial Sloan-Kettering cancer center (MSKCC) nomogram is the most representative (7). However, these models still have limitations. For instance, several postoperative pathologic characteristics were included in these nomograms, indicating that their use in intraoperative surgical volume decisions is difficult.

Inflammatory cells and mediators play a significant role in cancer progression, such as breast cancer (10). Recently, some combined inflammatory indicators have been reported to be associated with worse prognostic outcomes in various cancers (11). These inflammatory indicators have been applied to predict the recurrence, metastasis, prognosis, adjuvant therapy response, and molecular subtypes in breast cancer (12-16). A meta-analysis showed there were significant differences in the incidence of high levels of platelet-to-lymphocyte ratio (PLR) between patients with and without lymph nodes metastases [odds ratio (OR) =4.24, 95% confidence interval (CI): 2.73–6.59, Tau²<0.001, and I²=0.0%] (17). Yang et al. validated that the high level of PLR and vascular tumor thrombus are risk factors for SLN metastases (12). These inflammatory indicators may play an important role in the prediction of non-SLNs metastases.

The present study aims to develop and validate a nomogram based on the clinical characteristics and inflammatory indicators related to non-SLN metastasis to predict the risk of non-SLN metastases in patients with positive SLNs. This novel nomogram can help clinicians determine the extent of intraoperative resection in early breast cancer patients with positive SLNs. We present this article in accordance with the TRIPOD reporting checklist (available at https://gs.amegroups.com/article/view/10.21037/gs-22-585/rc).

Methods

Study population

The original study cohort included 489 patients with primary invasive breast cancer who underwent SLNB between December 2014 to August 2021 at Xuanwu Hospital, Capital Medical University. Our study population met the following criteria: (I) early-stage primary invasive breast carcinoma, stage cT1-2N0M0; (II) successful SLNB according to the fluorescence tracer and dye methods; (III) positive SLN intraoperatively; and (IV) completion of ALND with at least 10 nodes examined. Patients who received neoadjuvant chemo/radiotherapy treatment were excluded.

The validation cohort included patients from another treatment group in the same hospital during the same period, with the same inclusion criteria as the original series. A total of 46 patients were included in the external validation of the performance of the predictive tool.

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Committee of Xuanwu Hospital, Capital Medical University [No. (2020)009]. Individual consent for this retrospective analysis was waived, and the patient data were kept confidential.

SLNB procedure

SLNB was performed intraoperatively with a fluorescence tracer and nano-carbon dye. Before surgery, the nano-carbon dye was injected around the areola of the breast. After sterilization of the operating area, indocyanine green was injected into the subareolar tissue. After 2 to 5 minutes of massage, subcutaneous lymphatic drainage was observed on fluorescent images. Lymph nodes that were fluorescent or stained with the dye were accepted as SLNs.

The frozen SLN tissue was examined by hematoxylin and eosin (H&E) staining intraoperatively. Subsequently, complete ALND was performed on patients with positive SLNs. Postoperatively, routine H&E analysis was conducted for the remaining SLNs tissues and all additional nodes identified by ALND.

Measure of the tumor size

The resected tumor size was defined according to the widest diameter of the largest tumor, and the maximum section width was measured using calipers during the operation. The tumor size on ultrasound (US) and the pathologic tumor size were reported by authoritative ultrasonographers and pathologists.

Detection of inflammatory indicators in peripheral blood

Peripheral blood samples were collected before surgery, and the number of lymphocytes, neutrophil-to-lymphocyte ratio (NLR), PLR, systemic inflammation response index (SIRI), systemic immune-inflammation index (SII), and lymphocyte-to-monocyte ratio (LMR) were calculated according to the preoperative blood routine examination results.

Data processing

This was a retrospective study, and data were collected from the electronic medical record. Univariate analysis and the forward stepwise likelihood ratio method were applied to reduce the included independent variables to obtain stable regression coefficients in the logistic regression.

For the continuous variables, we excluded independent variables with extreme and missing values, preventing interference with the results. Furthermore, due to the study grouping, blinding for study subjects and medical staff was not available. Thus, we also excluded sensitive information, ensuring that the statistical analysts were blinded.

Statistical methods

A univariate analysis was performed to determine the risk factors associated with non-SLN metastases in patients with positive SLNs. The chi-square test or Fisher’s exact test was used for categorical variables, and the continuous variables were analyzed by Student’s t-test or the Mann-Whitney U test, as appropriate.

To identify the predictive factors of non-SLN metastases, all variables with a P value <0.1 in the univariate analysis were analyzed by multivariable logistic regression, using a forward stepwise likelihood ratio method. Only preoperative and intraoperative variables were considered in our regression. A nomogram model for analyzing the risk of non-SLN metastases was created based on the R package “rms” (version 6.2-0; Frank E. Harrell Jr, Email: fh@fharrell.com). The length of the line in the nomogram reflected the contribution of each factor to non-SLN metastases. The risk score was calculated via the R package “nomogramFormula” (version 1.2.0.0; Jing Zhang, Email: zj391120@163.com). The calibration curve was used to examine the predictive capability of the nomogram. The goodness-of-fit of the nomogram model was evaluated with the Hosmer-Lemeshow test using the R Package “ResourceSelection” (version 0.3-5; Peter Solymos, Email: solymos@ualberta.ca).

The nomogram model was then constructed. The accuracy of the models was measured using the area under the receiver operating characteristic (ROC) curve (AUC) with the R Package “pROC” (version 1.17.0.1; Xavier Robin, Email: pROC-cran@xavier.robin.name). The likelihood of non-SLN metastasis in the MSKCC model was obtained based on an online calculation tool (http://www.nomograms.org). The clinical decision value between the nomogram and MSKCC models was compared by decision curve analysis (DCA) using the R Package “rmda” (version 1.6; Marshall Brown, Email: mdbrown@fredhutch.org). The population prevalence of this case-control study was set to 50% based on previous research (2). A 10-fold cross-validation with 400 times repetitions was performed for training and testing our model caret via the R Package “caret” (version 6.0-88; Max Kuhn, Email: mxkuhn@gmail.com). Validation data were calculated by the nomogram, and its accuracy was expressed by the AUC.

Data analyses were performed using R software (version 4.4.1, R Foundation for Statistical Computing). In this study, a two-sided P value <0.05 was considered statistically significant.

Results

Patient characteristics

The original cohort enrolled 489 female breast cancer patients who underwent SLNB between December 2014 and August 2021. Among them, 96 patients (19.6%) were diagnosed as having positive SLNs during operation, including one patient with micrometastasis. The false negative rate in the frozen section was 0.41%, and two patients were reoperated after intraoperative false negative. Based on the exclusion criteria, a total of 83 patients were included in this study, and non-SLN metastases were observed in 42 of these patients (50.6%). The mean age of the patients was 56.4 (range, 33–87) years. The median size of the tumor was 2.00 (range, 1.50–2.65) cm. The average number of removed SLNs per single patient was 3±1.33. The clinical characteristics of the original and validation cohorts are presented in Tables 1,2. All of the patients were Chinese women.

Univariate and multivariate analyses and identification of risk factors

Univariate analysis demonstrated that ten variables were associated with non-SLN metastases, including age, type of surgery, number of positive SLNs, number of negative SLNs, resected tumor size, pathologic tumor size, lymphovascular invasion (LVI), SIRI, LMR, and monocyte count (all P<0.1, Table 1). The postoperative variables were excluded from the filtered variables. In the multivariable logistic regression analysis, age [odds ratio (OR) =0.171; 95% confidence interval (CI): 0.034 to 0.859; P=0.032], number of positive SLNs (OR =5.067; 95% CI: 1.290 to 19.91; P=0.020), number of negative SLNs (OR =0.550; 95% CI: 0.347 to 0.871; P=0.011), resected tumor size (OR =2.477; 95% CI: 1.052 to 5.830; P=0.038), and monocyte count (OR =1.009; 95% CI: 1.002 to 1.016; P=0.012) were associated with non-SLN metastases (Table 3).

Development and evaluation of nomogram

As shown in Figure 1, a nomogram model named five-factor was developed. The calibration curve was generated by 1,000 times resample via the bootstrap method for predicting the value of the nomogram for non-SLN metastases. The calibration curve exhibited good consistency between the actual observation and predicted probability (Figure 2). The Hosmer-Lemeshow test also showed non-significant goodness of fit in the nomogram (Chi-square =8.224, P=0.412).

Figure 1 Nomogram for predicting non-sentinel lymph node metastases in early breast cancer patients with positive sentinel lymph nodes. For each patient, five lines were drawn upward to determine the points received from the five predictors in the nomogram. The sum of these points is located on the ‘Total Points’ axis. SLN, sentinel lymph node; MONO, monocyte count.

Figure 2 Calibration curve for the five-factor nomogram, which shows excellent goodness-of-fit.

Validation of the nomogram model

The ROC curves were drawn in Figure 3. The five-factor model achieved an AUC of 0.867 (95% CI: 0.788–0.945, Table 4), whereas that of the MSKCC model was 0.754 (95% CI: 0.646–0.862, Table 4).

Figure 3 ROC curve for predicting non-sentinel lymph node metastases by the five-factor nomogram (A) and the MSKCC model (B). AUC, area under the receiver operating characteristics curve; MSKCC, Memorial Sloan Kettering Cancer Center; ROC, receiver operating characteristic.

The DCA of the models was drawn in Figure 4. The net benefit was defined as the benefit of true positives minus the harm of false positives (18). The risk threshold was a reference for treatment decisions, above which ALND would be performed. The model with a gray line represented that all patients would undergo ALND, while that with a black line represented that all patients would not undergo ALND. DCA showed high net benefits in both models among almost all threshold probabilities. The five-factor model exhibited a better clinical effect than the MSKCC model.

Figure 4 Decision curves of the five-factor nomogram and the MSKCC model. The x-axis represents the threshold probabilities, and the y-axis measures the net benefit calculated by adding the true positives and subtracting the false positives. The horizontal line along the x-axis assumes that non-sentinel lymph node metastases occurred in no patients, whereas the gray line assumes that all patients will have non-sentinel lymph node metastases at a specific threshold probability. The red line represents the net benefit of using the five-factor nomogram, while the blue line represents the MSKCC model. MSKCC, Memorial Sloan Kettering Cancer Center; ALND, axillary lymph node dissection.

To validate the nomogram model, we used 10-fold cross-validation with 400 times repetitions internally. The AUC of the five-factor nomogram model was still 0.830, indicating a stable predictive effect.

Each patient’s information from the external validation cohort was calculated using the five-factor nomogram to validate the predictive model. The ROC and calibration curves are displayed in Figure 5. The external validation AUC was 0.727 (95% CI: 0.5763–0.8779, Table 4), and the consistency between the actual observation and predicted probability was good. The accuracy of the prediction in patients who did not meet the criteria of the Z-0011 trial was also evaluated; the ROC curve and calibration curves are shown in Figure 6, and the AUC was 0.828 (95% CI: 0.7405–0.9154, Table 4).

Figure 5 ROC curve (A) and calibration curve (B) for the external validation cohort. AUC, area under the receiver operating characteristics curve; ROC, receiver operating characteristic.

Figure 6 ROC curve (A) and calibration curve (B) for all of the patients who did not meet the Z-0011 trial criteria. AUC, area under the receiver operating characteristics curve; ROC, receiver operating characteristic.

Discussion

After a series of clinical trials, SLNB has already replaced ALND as the standard procedure for patients with negative SLNs (4,19). With the increasing application of SLNB, scholars are paying more attention to patients with one or more positive SLNs. Among these patients, 53% have additional axillary non-SLN metastases, based on the findings of a large meta-analysis (2). This means the remaining patients undergo unnecessary ALND, without therapeutic benefit. A recent large study also validated that the 10-year OS for patients treated with SLNB alone was non-inferior to those treated with ALND, among women with early-stage invasive breast cancer with 1 or 2 SLN metastases (20). These trials confirmed that we may have a chance at further reducing the application of ALND for patients without additional non-SLN metastases.

Although several predictive models have aimed to evaluate the risk of non-SLN metastases, postoperative factors, including pathological tumor size, LVI, and histologic grade, were always included in these models (7-9,21,22). Among these, the nomogram developed by the MSKCC was the representative model, which has been verified in different countries (22-26). In our present study, a five-factor nomogram model was constructed to guide intraoperative decisions, based on a comparison with the MSKCC model and model validation.

According to the ROC curves, both the five-factor model and the MSKCC model exhibited a high predictive capability. The AUC of the MSKCC model was 0.754, which was close to the original MSKCC result of 0.75, which reflected the general adaptability of the model (7). Our five-factor model had a high AUC of 0.867, higher than that of the MSKCC model and other previous models. Thus, we explored the clinical value of the two models. DCA showed more value by examining model performance when there is no consensus regarding the risk threshold of non-SLN metastases (27). Compared with the MSKCC model, our nomogram had a better performance across a range of plausible risk thresholds and can be applied for intraoperative decision-making. After further internal cross-validation and external validation, the five-factor nomogram was confirmed as a reliable tool for intraoperatively predicting the likelihood of non-SLN metastasis in patients with positive SLNs.

The number of positive and negative SLNs were powerful predictive risk factors, which were included in nearly all previously published models (7-9,21,22). In our nomogram, the number of positive SLNs contributed significantly based on the length of the line in Figure 1. Age was also included in previous models as a protective predictor, which is consistent with our findings (21,22). In this study, age was divided into two groups based on the following reasons. On the one hand, neither age >50 years nor menopause status was confirmed as an independent predictor of non-SLN metastases in the previous models (21,22,28). On another hand, older patients would hardly tolerate ALND and its adverse effects, and omitting ALND may not affect their OS benefits. In our model, the size of the resected tumor was chosen as a predictive factor, instead of the pathological tumor size. It is easy for the resected size to be obtained intraoperatively, compared with the pathological size. We observed no significant differences in the resected positive SLN size. The accidental omission of micrometastasis in the frozen section was acceptable.

Local immune response and systemic inflammatory response contribute to tumor growth, invasion, and metastasis (29). The indicators in peripheral blood have been shown to reflect the inflammatory conditions of tumor cells (30). In this study, the LMR and monocyte count were statistically significant. LMR was calculated using the lymphocyte and monocyte counts [lymphocyte count/monocyte count (LMR)]. Lymphocytes play a critical role in tumor immune surveillance and anti-tumor effects (31-33). As for monocytes, different subpopulations may play opposite roles. Monocytes and macrophages are typically the first line of defense against tumor cells (34). However, once tumor cells escape the immune machinery, these cells turn to promote tumorigenesis and tumor progression (10). Goto et al. also found that the ratio of lymphocytes to monocytes is a protective prognostic factor for breast cancer patients (35). However, the mechanism through which the ratio of these indicators may impact breast cancer prognosis remains unclear. We hypothesized further amplifying the inflammatory response’s impact by calculating this ratio. From this point of view, the ratio of the indicators was predicted to be associated with metastasis. Various combinations of inflammatory indicators in peripheral blood may be novel tumor markers. The inflammatory indicators in peripheral blood could provide a cheap and simple method for breast cancer prognostication.

Several limitations to our study should be noted. Firstly, some potential factors may not have been discovered owing to the sample size, and further research with a larger sample size is needed. Secondly, the nomogram is merely an initial exploration with validation. We plan to begin multi-center external verifications to support the accuracy of our model. Thirdly, although the number of positive SLNs was the most powerful predictor, the nomogram could not be used if no SLNs are removed.

Conclusions

At present, there seems to be a tendency to omit ALND in early-stage breast cancer patients. We developed a novel nomogram for predicting non-SLN metastases in early-stage patients using preoperative and intraoperative factors. This nomogram model could assist clinicians to decide whether to perform ALND in early-stage breast cancer patients with positive SLNs.

Acknowledgments

We thank surgeons, patients, and family members who provided clinical information.

Funding: This work was supported by the Beijing Breast Disease Prevention and Treatment Society, Cancer Prevention and Treatment Research Projects, China (No. 2015-8-8).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://gs.amegroups.com/article/view/10.21037/gs-22-585/rc

Data Sharing Statement: Available at https://gs.amegroups.com/article/view/10.21037/gs-22-585/dss

Peer Review File: Available at https://gs.amegroups.com/article/view/10.21037/gs-22-585/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-22-585/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by Ethics Committee of Xuanwu Hospital, Capital Medical University [No. (2020)009]. Individual consent for this retrospective analysis was waived, and the patient data were kept confidential.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Rao R, Euhus D, Mayo HG, et al. Axillary node interventions in breast cancer: a systematic review. JAMA 2013;310:1385-94. [Crossref] [PubMed]
2. Kim T, Giuliano AE, Lyman GH. Lymphatic mapping and sentinel lymph node biopsy in early-stage breast carcinoma: a metaanalysis. Cancer 2006;106:4-16. [Crossref] [PubMed]
3. Shaw JH, Rumball EM. Complications and local recurrence following lymphadenectomy. Br J Surg 1990;77:760-4. [Crossref] [PubMed]
4. Krag DN, Anderson SJ, Julian TB, et al. Sentinel-lymph-node resection compared with conventional axillary-lymph-node dissection in clinically node-negative patients with breast cancer: overall survival findings from the NSABP B-32 randomised phase 3 trial. Lancet Oncol 2010;11:927-33. [Crossref] [PubMed]
5. Gradishar WJ, Moran MS, Abraham J, et al. NCCN Guidelines® Insights: Breast Cancer, Version 4.2021. J Natl Compr Canc Netw 2021;19:484-93. [Crossref] [PubMed]
6. Donker M, van Tienhoven G, Straver ME, et al. Radiotherapy or surgery of the axilla after a positive sentinel node in breast cancer (EORTC 10981-22023 AMAROS): a randomised, multicentre, open-label, phase 3 non-inferiority trial. Lancet Oncol 2014;15:1303-10. [Crossref] [PubMed]
7. Van Zee KJ, Manasseh DM, Bevilacqua JL, et al. A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol 2003;10:1140-51. [Crossref] [PubMed]
8. Meretoja TJ, Leidenius MH, Heikkilä PS, et al. International multicenter tool to predict the risk of nonsentinel node metastases in breast cancer. J Natl Cancer Inst 2012;104:1888-96. [Crossref] [PubMed]
9. Reynders A, Brouckaert O, Smeets A, et al. Prediction of non-sentinel lymph node involvement in breast cancer patients with a positive sentinel lymph node. Breast 2014;23:453-9. [Crossref] [PubMed]
10. Mantovani A, Allavena P, Sica A, et al. Cancer-related inflammation. Nature 2008;454:436-44. [Crossref] [PubMed]
11. Ethier JL, Desautels D, Templeton A, et al. Prognostic role of neutrophil-to-lymphocyte ratio in breast cancer: a systematic review and meta-analysis. Breast Cancer Res 2017;19:2. [Crossref] [PubMed]
12. Yang L, Wang H, Ma J, et al. Association between the platelet to lymphocyte ratio, neutrophil to lymphocyte ratio and axillary lymph node metastasis in cT1N0 breast cancer patients. Am J Transl Res 2021;13:1854-61. [PubMed]
13. Zenan H, Zixiong L, Zhicheng Y, et al. Clinical prognostic evaluation of immunocytes in different molecular subtypes of breast cancer. J Cell Physiol 2019;234:20584-602. [Crossref] [PubMed]
14. Kim JY, Jung EJ, Kim JM, et al. Dynamic changes of neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio predicts breast cancer prognosis. BMC Cancer 2020;20:1206. [Crossref] [PubMed]
15. Graziano V, Grassadonia A, Iezzi L, et al. Combination of peripheral neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio is predictive of pathological complete response after neoadjuvant chemotherapy in breast cancer patients. Breast 2019;44:33-8. [Crossref] [PubMed]
16. Vernieri C, Mennitto A, Prisciandaro M, et al. The neutrophil-to-lymphocyte and platelet-to-lymphocyte ratios predict efficacy of platinum-based chemotherapy in patients with metastatic triple negative breast cancer. Sci Rep 2018;8:8703. [Crossref] [PubMed]
17. Zhang M, Huang XZ, Song YX, et al. High Platelet-to-Lymphocyte Ratio Predicts Poor Prognosis and Clinicopathological Characteristics in Patients with Breast Cancer: A Meta-Analysis. Biomed Res Int 2017;2017:9503025. [Crossref] [PubMed]
18. Van Calster B, Wynants L, Verbeek JFM, et al. Reporting and Interpreting Decision Curve Analysis: A Guide for Investigators. Eur Urol 2018;74:796-804. [Crossref] [PubMed]
19. Giuliano AE, Kirgan DM, Guenther JM, et al. Lymphatic mapping and sentinel lymphadenectomy for breast cancer. Ann Surg 1994;220:391-8; discussion 398-401. [Crossref] [PubMed]
20. Giuliano AE, Ballman KV, McCall L, et al. Effect of Axillary Dissection vs No Axillary Dissection on 10-Year Overall Survival Among Women With Invasive Breast Cancer and Sentinel Node Metastasis: The ACOSOG Z0011 (Alliance) Randomized Clinical Trial. JAMA 2017;318:918-26. [Crossref] [PubMed]
21. Bevilacqua JL, Kattan MW, Fey JV, et al. Doctor, what are my chances of having a positive sentinel node? A validated nomogram for risk estimation. J Clin Oncol 2007;25:3670-9. [Crossref] [PubMed]
22. Chen JY, Chen JJ, Yang BL, et al. Predicting sentinel lymph node metastasis in a Chinese breast cancer population: assessment of an existing nomogram and a new predictive nomogram. Breast Cancer Res Treat 2012;135:839-48. [Crossref] [PubMed]
23. Specht MC, Kattan MW, Gonen M, et al. Predicting nonsentinel node status after positive sentinel lymph biopsy for breast cancer: clinicians versus nomogram. Ann Surg Oncol 2005;12:654-9. [Crossref] [PubMed]
24. Klar M, Foeldi M, Markert S, et al. Good prediction of the likelihood for sentinel lymph node metastasis by using the MSKCC nomogram in a German breast cancer population. Ann Surg Oncol 2009;16:1136-42. [Crossref] [PubMed]
25. Cho J, Han W, Lee JW, et al. A scoring system to predict nonsentinel lymph node status in breast cancer patients with metastatic sentinel lymph nodes: a comparison with other scoring systems. Ann Surg Oncol 2008;15:2278-86. [Crossref] [PubMed]
26. Gur AS, Unal B, Ozbek U, et al. Validation of breast cancer nomograms for predicting the non-sentinel lymph node metastases after a positive sentinel lymph node biopsy in a multi-center study. Eur J Surg Oncol 2010;36:30-5. [Crossref] [PubMed]
27. Kerr KF, Brown MD, Zhu K, et al. Assessing the Clinical Impact of Risk Prediction Models With Decision Curves: Guidance for Correct Interpretation and Appropriate Use. J Clin Oncol 2016;34:2534-40. [Crossref] [PubMed]
28. Ishizuka Y, Horimoto Y, Nakamura M, et al. Predictive Factors for Non-sentinel Nodal Metastasis in Patients With Sentinel Lymph Node-positive Breast Cancer. Anticancer Res 2020;40:4405-12. [Crossref] [PubMed]
29. Diakos CI, Charles KA, McMillan DC, et al. Cancer-related inflammation and treatment effectiveness. Lancet Oncol 2014;15:e493-503. [Crossref] [PubMed]
30. Chen L, Kong X, Yan C, et al. The Research Progress on the Prognostic Value of the Common Hematological Parameters in Peripheral Venous Blood in Breast Cancer. Onco Targets Ther 2020;13:1397-412. [Crossref] [PubMed]
31. Denkert C, Loibl S, Noske A, et al. Tumor-associated lymphocytes as an independent predictor of response to neoadjuvant chemotherapy in breast cancer. J Clin Oncol 2010;28:105-13. [Crossref] [PubMed]
32. Gregory AD, Houghton AM. Tumor-associated neutrophils: new targets for cancer therapy. Cancer Res 2011;71:2411-6. [Crossref] [PubMed]
33. Swann JB, Smyth MJ. Immune surveillance of tumors. J Clin Invest 2007;117:1137-46. [Crossref] [PubMed]
34. Gouveia-Fernandes S. Monocytes and Macrophages in Cancer: Unsuspected Roles. Adv Exp Med Biol 2020;1219:161-85. [Crossref] [PubMed]
35. Goto W, Kashiwagi S, Asano Y, et al. Predictive value of lymphocyte-to-monocyte ratio in the preoperative setting for progression of patients with breast cancer. BMC Cancer 2018;18:1137. [Crossref] [PubMed]

(English Language Editor: A. Kassem)