Roles of artificial intelligence and high frame-rate contrast-enhanced ultrasound in the differential diagnosis of Breast Imaging Reporting and Data System 4 breast nodules

Introduction

Breast cancer is a significant global health concern for women (1), and its morbidity and mortality rates continue to increase worldwide (2,3). Breast cancer has become one of the most serious global disease burdens due to the lack of efficient early screening and detection methods and the absence of cost-effective treatments (4). The fifth edition of the Breast Imaging Reporting and Data System (BI-RADS), released by the American College of Radiology (ACR), divides breast lesions into six categories, among which BI-RADS 4 lesions have a probability of malignancy ranging from 2% to 95% (5). In addition, this category has three subcategories. Unfortunately, the benign or malignant features of lesions in any subcategory are not clearly defined on conventional ultrasound (CUS) and often overlap with each other, which makes it difficult to accurately determine the benign or malignant nature of a BI-RADS 4 lesion. Needle biopsy is often required for the pathological diagnosis. However, biopsy is an invasive procedure that may lead to postoperative complications and instability. Additionally, excessive needle biopsies can increase the economic burden placed on patients. Further, the accuracy (ACC) of the specimen collection at the lesion tissues will also affect the diagnostic results (6). Thus, a non-invasive and accurate differential diagnosis of ACR BI-RADS 4 breast nodules before surgery is of great clinical significance.

The early diagnosis of breast cancer relies mainly on imaging examinations including doppler ultrasonography (DUS), mammography, and magnetic resonance imaging (MRI). Mammography is a commonly used diagnostic tool for breast cancer due to its high repeatability/resolution and non-invasiveness. It can accurately characterize the morphology of breast lesion edges, determine the density of breast tissue, and visualize microcalcifications. However, in certain special circumstances, there is a certain rate of missed diagnosis, such as deep lesion location and early absence of microcalcifications (7,8). In addition, radiation exposure during mammography also limits its clinical use (9-12).

MRI is capable of accurately identifying soft tissues and presenting tumor lesions in a multi-image and multi-directional manner without causing radiation damage to patients. However, its application has some limitations. In the early stages of breast cancer, MRI is less sensitive to common microcalcifications, and its image quality can be easily affected by factors such as respiratory artifacts and heartbeat, leading to severely distorted or abnormally altered image signals. Further, MRI is not suitable for large-scale population screening due to its high cost (13). Contrast-enhanced spectral mammography (CESM) is also an effective method for diagnosis of breast cancer, with high diagnostic sensitivity. A study has shown that the diagnostic performance of CESM seems to be more effective than MRI (14). Not to be underestimated is the value of dual-energy CT (DECT) in the diagnosis of breast lesions, which, if proven reliable for local staging, would allow the evaluation of primary breast lesions, lymph node status and search for distant metastases in a single diagnostic examination. However, there are many disadvantages that include the radiation dose, which can be high in case of multiple post-contrast phases, and the poor accuracy in the evaluation of carcinomas in situ, compared to MRI (15).

Ultrasonography is widely used in the differential diagnosis of benign and malignant breast lesions as a non-invasive, real-time, dynamic, reproducible, and inexpensive tool. However, the diagnostic performance of CUS is unsatisfactory. Contrast-enhanced ultrasound (CEUS) is a pure blood-pool imaging technique. The use of contrast agent is limited by size, as the agent can only move inside the blood vessels and cannot enter the tissue through the blood vessel wall. By tracking the movement trajectory of the contrast agent in the blood vessels, US imaging can reflect the microcirculation perfusion of the agent in the lesion and the surrounding tissues, and the number, shape, spatial distribution, and other anatomical and morphological characteristics. Further, the enhanced mode can indirectly reflect the hemodynamic features. Thus, CEUS has unique advantages in distinguishing between benign and malignant lesions (16). Compared with CUS, CEUS can dramatically increase the ACC of breast lesion diagnoses (17,18). Recent studies have shown that CEUS has a diagnostic performance similar to or even better than that of contrast-enhanced MRI in the diagnosis of benign and malignant breast lesions (19,20). The second-generation “pure blood-pool” contrast agent for CEUS, such as the one by SonoVue, produces microbubbles that enter the breast tissue and capillary network at the lesion site; scanning in contrast-enhanced mode can clearly display the microcirculatory blood perfusion in the lesion and surrounding glands, aiding in the diagnosis and differentiation of breast cancer (21,22).

The frame rate (FR), which is an important parameter in CEUS, refers to the frequency at which images are continuously sampled in frames [frames per second (fps)]. A low FR reduces temporal resolution and negatively affects the real-time visualization of lesions, while a high FR increases the disruption of microbubbles due to the emission of US pulses, thus reducing imaging time (23)]. High frame-rate contrast-enhanced ultrasound (HiFR-CEUS) is an emerging imaging technology, with an average FR of >30 fps (which can be compared to a FR of about 10 fps for traditional CEUS). Using an innovative high-speed computing ONE Sonography^® Technology+ (ZST+) platform, it can improve the trajectory path of microbubbles in blood vessels to display the morphological structure of blood vessels (upcoming >60 fps). With high temporal resolution, HiFR-CEUS can track the movement of microbubbles in the arterial phase, portal venous phase, and delayed phase (or extrahepatic venous phase) in a real-time fashion, thus enabling the detection and etiologic diagnosis of lesions.

US Smart Breast technology is one of the computer-aided diagnostic technologies developed by Mindray, China. Based on the convolutional neural network (CNN), a deep-learning (DL) algorithm, it is intelligent analysis software for breast lesions that combines both the training results of a large number of clinical data, and the findings of a BI-RADS automatic analysis. It can automatically identify breast nodules, as it is able to automatically detect and analyze the relevant information of breast nodules through DL models and evaluate the features (e.g., size, shape, edges, internal echo characteristics, and blood flow signals) of nodules according to the BI-RADS lexicon, thus automatically distinguishing between a benign or malignant nodule (as “possibly benign” or “possibly malignant”).

Since the past few years, there has been a wave of research on the use of artificial intelligence (AI) in breast lesions (24-26). In relation to the use of AI in breast US, a major interest is to distinguish between benign and malignant breast masses based on B-mode features according to the findings of breast imaging and BI-RADS. Shen et al. (27) evaluated eight BI-RADS US computerized features, including shape, orientation, margin, lesion boundary, echo pattern, and posterior acoustic feature classes, to distinguish among breast masses, and found that these features significantly differed between malignant and benign lesions. A very high correlation was found between the angular characteristic and pathological result. Based on 7,408 images acquired in the United States, Han et al. (28) developed a CNN DL framework to distinguish between malignant breast lesions and benign nodules. Their networks were able to classify malignant lesions with an ACC of 90% within a short period. Thus, the proposed method could work in tandem with human radiologists to improve diagnostic performance. Kim et al. (29) evaluated the ability of a tool called “S-Detect” to distinguish among breast lesions when applied to breast ultrasonography. When the cutoff was set at category 4a, the diagnostic ACC and area under the curve (AUC) for the receiver operator characteristic (ROC) curve of the S-Detect were significantly higher than those of a radiologist who was a specialist in breast imaging. Based on the multi-section grayscale US images and pathology images of 693 breast cancer lesions, Huang et al. (30) used a multi-view attention network and a multi-instance learning-based attention model to extract the US and pathology image features, respectively. Notably, an US-guided co-attention module was designed for feature fusion. The AUCs of the US image pathomic models in distinguishing between luminal and non-luminal early breast cancers ranged from 0.90 to 0.93. US elastography is an adjunct to the conventional B-mode US and has been shown to be useful in lesion characterization. Barr (31) established a DL architecture that is capable of automatically extracting features obtained from elastography.

The US-based diagnostic criteria for breast lesions depend on the establishment of US BI-RADS, which enables the better assessment of the characterization of breast lesions. CEUS is a safe and reliable novel US technology that does not involve ionizing radiation. CEUS has been shown to dynamically reveal the blood perfusion of breast lesions in real time, resulting in a significant increase in the detection rate of breast masses, and has the ability to distinguish between benign and malignant masses (32). On this basis, HiFR-CEUS increases the sampling FR, increases the sampling data, and prolongs the visualization of lesion perfusion within a limited time window (the interval between the arterial and venous phases), which enables the movement trajectory of microbubbles in the blood vessels (i.e., the feature information of lesion perfusion) to be more accurately tracked.

US Smart Breast technology is a computer-aided diagnostic technology that can automatically identify breast lesions. After the relevant sections are collected according to the instructions, the system automatically analyzes the size, shape, edges, internal echo characteristics, and internal blood flow characteristics of the breast nodule for comprehensive evaluation, and a diagnosis of “possibly benign” or “possibly malignant” is made accordingly. This technology offers a new examination method for the benign and malignant identification of breast nodules and may avoid the misdiagnosis caused by the subjective errors of the radiologists. Our present study was designed to explore the values of AI (US Smart Breast technology) and/or HiFR-CEUS in evaluating benign and malignant BI-RADS 4 nodules, and thus inform the development of clinical treatment protocols. We present this article in accordance with the STARD reporting checklist (available at https://gs.amegroups.com/article/view/10.21037/gs-24-187/rc)

Methods

Subjects

Patients with BI-RADS 4 breast nodules who were hospitalized at the Department of Thyroid and Breast Surgery, Taizhou People’s Hospital from December 2021 to June 2022 were enrolled in the study. The lesions were diagnosed by routine breast US at our hospital or other centers. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by Ethics Committee of Taizhou People’s Hospital (No. KY202107303) and informed consent was taken from all the patients.

Inclusion criteria

To be eligible for inclusion in this study, patients had to meet the following inclusion criteria: (I) have BI-RADS 4 lesions identified on two-dimensional US; (II) voluntarily participate in this study and sign the informed consent forms; and (III) have complete clinicopathological and radiological data.

Exclusion criteria

Patients were excluded from the study if they met any of the following exclusion criteria: (I) had heart, liver, kidney, lung, and/or other organ dysfunction; (II) experienced adverse reactions to contrast materials; (III) had a concomitant malignancy at other site(s); (IV) had chest wall or skin problems affecting scanning; (V) were lactating or pregnant women; and/or (VI) had communication difficulties.

Grouping

The included subjects were divided into the AI (Smart Breast) group, HiFR-CEUS group, and combination (both AI and HiFR-CEUS) group.

Instrument and agents

Both the CUS and AI (Smart Breast) examinations used the linear array probe L14-3 WU of the Mindray R9 US diagnostic instrument with a probe frequency of 4–15 MHz.

During HiFR-CEUS, the linear array probe L14-3 WU of the Mindray R9 US diagnostic instrument (Mindray) was used with a probe frequency of 9 MHz; the SonoVue contrast agent (Bracco, Italy) was used, to which 5 mL of normal saline was added to prepare a microbubble suspension by repeated vibration.

Examinations

CUS examination

The patient was asked to take a supine position, with two hands around the head and elbows abducted to fully expose the bilateral breasts and bilateral axillae. The CUS scan of the breast was performed first to obtain grayscale two-dimensional images. The scans began directly above the breast (at 12 o’clock), followed by a comprehensive scan of the breast clockwise. After the nodule was located, the characteristics of the nodule (e.g., size, shape, edges, and internal echoes) were recorded to determine whether it was a BI-RADS 4 lesion. BI-RADS 4 lesions were included in this study, and the US diagnostic instrument was adjusted so that the nodule was clearly displayed in the center of the screen.

AI

The US diagnostic instrument was adjusted to the Smart Breasts mode. The cross-section, longitudinal section, sections with possibly malignant signs (e.g., angulation and calcification), and the section with the richest blood flow were collected according to the instructions. The system automatically extracted the characteristics of the nodule and carried out an intelligent analysis to obtain the AI-based diagnosis of the nodule (i.e., “possibly benign” or “possibly malignant”).

HiFR-CEUS

• First, 25 mg of SonoVue (sulfur hexafluoride microbubbles, containing 59 mg of sulfur hexafluoride) (Bracco) (approval No. H20110350) was added to 5 mL of normal saline to prepare a microbubble suspension by repeated vibration.
• Under the CUS mode, the nodule was clearly displayed in the center of the screen, and the machine was then switched to the CEUS mode. The assistant used a 5.0-mL sterile syringe to aspirate 2.0 mL of the prepared suspension for rapid bolus injection through the elbow vein. Meanwhile, the sonographer clicked the “double real-time” and “backward save” buttons and selected the HiFR-CEUS mode, adjusted the mechanical index to less than 0.1, and adjusted the gain to ensure the optimal display of the target nodule. During the examination, the sonographer observed the entire dynamic perfusion process of the lesion under the HiFR-CEUS mode. The observation lasted for at least 1 minute with the probe fixed, followed by a dynamic scan of the entire lesion until the contrast agent subsided. The whole imaging process was saved dynamically.
• After the examination was completed, the dynamic HiFR-CEUS images were retrieved, and the characteristics of the lesions were analyzed and recorded by two senior sonographers to determine whether the nodule was benign or malignant.

Pathological examination

Specimens obtained by surgical excision or core-needle biopsy were made into sections for hematoxylin and eosin (H&E) staining (or immunohistochemical staining if necessary).

Image analysis and diagnostic methods

AI group

The Smart Breasts system automatically identified and located breast nodules, extracted the characteristic information of the nodules on the collected sections based on the BI-RADS lexicon, conducted a comprehensive assessment of the breast nodules, and automatically diagnosed the nodules (as “possibly benign” or “possibly malignant”).

HiFR-CEUS group

The main measures (33) included:

• Time to lesion enhancement (i.e., enhancement time), which was divided into rapid entry, synchronous entry along with surrounding normal tissues, and slow entry.
• Intensity of the lesion enhancement at the peak (i.e., enhancement intensity), which was divided into hyperenhancement (the lesion was significantly enhanced compared with the surrounding normal tissue), isoenhancement (the lesion was enhanced to the same degree as the surrounding normal tissue), hypoenhancement (the enhancement of the lesion was lower than that of the surrounding normal tissue), or no enhancement (no contrast agent entered the lesion).
• Sequence of perfusion of contrast agent into the lesion (i.e., enhancement sequence), which was divided into centripetal enhancement (the contrast agent was perfused from the edge to the center) and non-centripetal enhancement.
• Edges of the lesion after enhancement (i.e., whether the boundary with the surrounding normal tissue was clearly discernible), which was classified as clear or hard to discern.
• Morphology of the lesions after enhancement, which was classified as regular or irregular.
• Uniformity of the lesion (i.e., whether the lesion was uniformly enhanced), which was divided into uniform or uneven.
• Change in the size of the lesion after enhancement compared with the size measured under CUS, such that the lesion was classified as enlarged (the diameter of the lesion was increased by 2 mm or more compared with that before the imaging), or not enlarged.
• Presence or absence of a filling defect area in the lesion after enhancement, which was classified as “yes” (CEUS showed the presence of a local non-enhanced area in the lesion, but there was no non-enhanced lesion as a whole or CUS showed the fluid area in the lesion and a wide acoustic and shadow area) or “no”.
• Presence or absence of a “crab-foot sign” in the lesion (i.e., radial enhancement along the edges of the lesion) after enhancement, which was classified as “yes” or “no”.
• Presence or absence of large feeding vessels around the lesion after enhancement (i.e., large or distorted blood vessels penetrated the lesion via the periphery of the lesion), which was classified as “yes” or “no”.

Diagnostic indicators (34)

Three types of benign lesions were diagnosed by HiFR-CEUS:

• Hyperenhanced nodules: there was no obvious change in the extent of the lesions after enhancement; the morphology remained regular, and there were no feeding vessels in the periphery.
• Uniformly enhanced nodules: the edges and morphology were hard to discern, there was no filling defect area, and there were no peripheral feeding vessels.
• Hypoenhanced nodules: the extent of the lesion remained unchanged or narrowed after enhancement, and there was no feeding vessel.

Three types of malignant lesions were diagnosed by HiFR-CEUS:

• Hyperenhanced nodules with an increase in the extent of enhancement that was larger than that before enhancement (both the transverse and longitudinal diameters were greater than 2 mm), with or without irregular morphology.
• Hyperenhanced nodules with increased centripetality, and internal filling defects, with or without an increase in the extent of the lesion after enhancement.
• Hyper- or iso-enhanced nodules with peripheral feeding vessels or the crab-foot sign, with or without internal filling defects.

When two sonographers involved in HiFR-CEUS disagreed about the sonographic characteristics of a specific lesion or over a diagnosis, a senior physician was consulted to decide whether the nodule should be assessed as possibly benign or malignant under HiFR-CEUS.

Combination group

The lesions were classified as malignant if either AI or HiFR-CEUS suggested the lesion was malignant.

Statistical analysis

The statistical analysis was performed using the SPSS 25.0 software package. The measurement data were expressed as the mean ± standard deviation, and the counting data in the four-fold contingency table were presented as frequencies. The sensitivity (SE), specificity (SP), ACC, positive predictive value (PPV), and negative predictive value (NPV) of the BI-RADS 4 breast nodules were calculated for the AI, HiFR-CEUS, and combination groups, respectively, and the pathological results obtained by surgery or needle biopsy served as the “gold standard”. The results were compared using McNemar’s exact test for paired proportions. The reliability of the three diagnostic methods was compared using a Kappa agreement analysis with the “gold standard” (i.e., the pathological diagnosis) (a Kappa value <0.40 indicated poor agreement, a value of 0.40–0.75 indicated fair agreement, and a value >0.75 indicated excellent agreement. A Pearson correlation analysis was conducted to examine the relationship between AI and HiFR-CEUS, and the ROC curves of the three diagnostic methods were plotted, and the differences in the AUCs were compared using the Z test.

Results

Pathological diagnoses

The pathological results obtained after the surgical resection of the breast nodules or core-needle biopsy were used as the “gold standard” for diagnosis. A total of 80 BI-RADS 4 nodules were collected, among which 62 were malignant, including invasive ductal carcinoma with intraductal carcinoma (n=1), intraductal carcinoma in situ (n=3), mucinous carcinomas (n=3), and invasive ductal carcinomas (n=55), and 18 were benign, including inflammatory lesions (n=2), intraductal papilloma (n=1), fibroadenosis with fibroadenomas (n=1), fibroadenosis (n=4), and fibroadenomas (n=10) (Table 1).

Comparisons of the three diagnostic methods with pathology

Examination results

Among the 62 malignant lesions, 47 were correctly diagnosed and 15 were misdiagnosed in the AI group, 46 were correctly diagnosed and 16 were misdiagnosed in the HiFR-CEUS group, and 61 were correctly diagnosed and one was misdiagnosed in the combination group. Among the 18 benign lesions, 17 were correctly diagnosed and one was misdiagnosed in the AI group, 17 were correctly diagnosed and one was misdiagnosed in the HiFR-CEUS group, and 16 were correctly diagnosed and two were misdiagnosed in the combination group (Table 2, and Figures 1,2).

Figure 1 A 43-year-old woman with a lump in the right breast. (A) AI suggested “possibly malignant”. (B) HiFR-CEUS showed that the nodule was hyperenhanced and had feeding vessels, and the diagnosis was also “possibly malignant”. (C,D) The pathologic diagnosis was invasive breast carcinoma, and under 100× magnification, H&E/IHC staining. The results of the enzyme linked immunosorbent assay were as follows: AR (2+), ER (2+, 50%), FR (1+, 10%), Cerb-2 (2+), CK14 (−), CK5/6 (−), CK34βE12 (+), E-cad (3+), EGFR (−), P120 (+), P53 (+), and Ki-67 (about 60%). AI, artificial intelligence; H&E, hematoxylin and eosin; HiFR-CEUS, high frame-rate contrast-enhanced ultrasound; IHC, immunohistochemistry.

Figure 2 A 65-year-old female with a lump in the left breast. (A) AI suggested a diagnosis of “possibly malignant”. (B) HiFR-CEUS showed that the nodule was hyperenhanced, with peripheral feeding vessels and internal filling defects, and the diagnosis was also “possibly malignant”. (C,D) The pathologic diagnosis was invasive breast carcinoma, and under 100× magnification, H&E/IHC staining. The results of the ELISA were as follows: AR (+), ER (−), FR (−), Cerb-2 (3+), E-cad (+), EGFR (−), P120 (+), CK14 (−), CK5/6 (−), P63 (−), P53 (+), and Ki-67 (about 35%). AI, artificial intelligence; H&E, hematoxylin and eosin; HiFR-CEUS, high frame-rate contrast-enhanced ultrasound; IHC, immunohistochemistry.

Comparison of the diagnostic performance of the three diagnostic methods

Both AI and HiFR-CEUS can be used to distinguish between benign and malignant BI-RADS 4 breast nodules. The combination group had better diagnostic performance than the AI and HiFR-CEUS group in terms of the SE, ACC, and NPV (Table 3).

The SE was significantly higher in the combination group than the AI and HiFR-CEUS groups (both P<0.05); however, no statistically significant difference was observed between the AI and HiFR-CEUS groups in terms of the SE (P>0.05). The SP was similar between the AI and HiFR-CEUS groups (P>0.05), but the SP of the AI and HiFR-CEUS groups was significantly higher than that of the combination group (both P<0.05). The ACC was significantly higher in the combination group than the AI and HiFR-CEUS groups (both P<0.05), but no statistically significant difference was found between the AI and HiFR-CEUS groups in terms of the ACC (P>0.05).

The PPV was highest in the AI group, followed by the HiFR-CEUS group, and the combination group; however, the differences were not statistically significant (all P>0.05). The NPV was significantly higher in the combination group than the AI and HiFR-CEUS groups (both P<0.05), but no statistically significant difference was found between the AI and HiFR-CEUS groups in terms of the NPV (P>0.05).

To sum up, compared with the those of the AI and HiFR-CEUS groups, the SE, ACC, and NPV of the combination group were significantly higher and the SP was significantly lower (all P<0.05), but no significant difference was found between the groups in terms of the PPV (P>0.05). Thus, the SE, ACC, and NPV of the combination group was higher.

There were no significant differences in the above diagnostic indicators between the AI and HiFR-CEUS groups (all P>0.05), which suggests that diagnostic performance of these two methods is comparable.

Reliability of the three diagnostic methods

Using the pathologic diagnosis as the “gold standard”, we found that AI and HiFR-CEUS were moderately consistent with the “gold standard” (Kappa value: 0.40–0.75), while the combination method had high agreement with the “gold standard” (Kappa value: >0.75), which indicates that the method that combined AI and HiFR-CEUS had the highest reliability in distinguishing between benign and malignant BI-RADS 4 breast nodules (Table 4).

AUCs of the three diagnostic methods

The results of the Pearson correlation analysis revealed a positive correlation between the AI and HiFR-CEUS groups (r=0.249, P<0.05). Three ROC curves were plotted for the AI, HiFR-CEUS, and combination groups, with the SE of each method for diagnosing 80 lesions as the ordinate and the 1-SP as the abscissa (Figure 3). The AUCs of these three diagnostic methods were 0.851±0.039, 0.815±0.047, and 0.936±0.039, respectively. The curve of the combination group was closest to the upper left corner of the graph. The AUC of the combination group was significantly higher than those of the AI and HiFR-CEUS groups (Z1=2.207, Z2=2.477, respectively, P<0.05), which indicates that these two methods combined had higher diagnostic value than AI alone or HiFR-CEUS alone in differentiating between benign and malignant BI-RADS 4 nodules. The AI group had a higher AUC than the HiFR-CEUS group, but the difference was not statistically significant (Z3=2.477, P<0.05). Therefore, they had comparable diagnostic performance.

Figure 3 ROC curves of AI, HiFR-CEUS, and combination groups in distinguishing between benign and malignant BI-RADS 4 breast nodules. AI, artificial intelligence; BI-RADS, Breast Imaging Reporting and Data System; HiFR-CEUS, high frame-rate contrast-enhanced ultrasound; ROC, receiver operator characteristic.

Discussion

As the most common malignancy among women worldwide, breast cancer has become one of the major public health challenges due to its high prevalence and mortality rates. As a radiation-free, reproducible, and affordable tool, US has been widely applied in the examination of various breast diseases. BI-RADS is a classification system for the US diagnosis of breast diseases and standardizes the sonographic diagnosis of breast lesions. BI-RADS 4 lesions can be further divided into three subcategories and have a probability of malignancy ranging from 2% to 95%. Lesions in different subcategories have shared or overlapping benign or malignant features, making them difficult to distinguish by CUS. Moreover, during a conventional two-dimensional US examination, the benign or malignant nature of a breast nodule is mainly judged according to the morphology, edges, location, internal echoes, and other characteristics on the sonograms, which largely depends on the subjective judgment of the sonographer and is thus greatly affected by the experience and expertise of the sonographer and other subjective factors. In recent years, many new US technologies (e.g., elastography, AI, automatic breast full volume scanning, CEUS, and Firefly technology) have provided more detailed and accurate diagnostic information for breast diseases and have provided theoretical bases for the differential diagnosis of breast cancer. US Smart Breast technology is a computer-aided diagnostic technology that can automatically identify breast lesions and automatically analyze the size, shape, edges, internal echo characteristics, and internal blood flow characteristics of the breast nodule for comprehensive evaluation, and a diagnosis of “possibly benign” or “possibly malignant” is made accordingly. HiFR-CEUS can accurately display the blood perfusion characteristics of breast lesions, providing a basis for the differential diagnosis of benign and malignant breast lesions. In the present study, we explored the value of AI and HiFR-CEUS in evaluating benign and malignant BI-RADS 4 nodules. Our findings will inform the development of clinical treatment protocols.

Value of US Smart Breast technology in the differential diagnosis of BI-RADS 4 breast nodules

AI has been increasingly applied in medical imaging in recent years. AI algorithms, particularly DL algorithms, have dramatically facilitated the advancement of medical imaging (35). AI systems are primarily used to identify and segment regions of interest (ROIs), extract features, and classify benign and malignant lesions in breast US imaging (36,37). At present, the classification of lesions in breast US images is mainly based on the BI-RADS system. Accordingly, AI systems with benign and malignant classification functions have gradually been developed to enable doctors with varying levels of experience to draw consistent conclusions. Becker et al. analyzed 637 breast US images (84 malignant and 553 benign lesions) using DL software. A randomly chosen subset of the images (n=445, 70%) was used for the software training, and the remaining images (n=192) were used for the model validation. Three readers with variable expertise also evaluated the validation set (a radiologist, resident, and medical student). It was found that the neural network, which was trained on only a few hundred cases, exhibited comparable ACC to the radiologist and better performance than the student (38). Thus, AI can efficiently assist in the classification and diagnosis of benign and malignant breast lesions.

In the present study, the US Smart Breast technology misdiagnosed a benign nodule (which was pathologically confirmed to be an inflammatory mass) as malignant. Due to the large extent and irregular shape of the lesion, Smart Breast might not have been able to accurately delineate the normal edges of the lesion, or precisely identify its margin information. In addition, 15 malignant nodules (including three pathologically confirmed intraductal carcinomas and 12 invasive ductal carcinomas) were misdiagnosed as benign, all of which were <10 mm in diameter, with clear edges and a regular shape. Three lesions also had internal microcalcifications. These misdiagnoses might be explained by the small sizes and the inability of Smart Breast to identify internal microcalcifications or internal blood flow signals.

US Smart Breast technology is highly valuable in the differential diagnosis of BI-RADS 4 breast nodules; however, its clinical application still has some limitations. First, microcalcification is a robust marker of breast cancer, and microcalcification detection is of great significance in the early diagnosis and treatment of breast cancer (39). Unfortunately, US Smart Breast technology cannot efficiently identify microcalcifications in breast nodules. Second, US Smart Breast technology is a computer-aided diagnostic technology that can automatically identify breast lesions and automatically analyze the size, shape, edges, internal echo characteristics, and internal blood flow characteristics of the breast nodule for comprehensive evaluation to make a diagnosis of “possibly benign” or “possibly malignant”. However, such a diagnosis does not combine information from elasticity assessments or blood perfusion information in the nodules. Third, the limited sample size and straightforward pathological findings might have introduced statistical biases, and our findings need to be validated in studies with larger sample sizes.

Value of HiFR-CEUS in the differential diagnosis of BI-RADS 4 breast nodules

Breast cancer is an angiogenesis-dependent disease, and its pathogenesis and progression are inseparable from tumor neovascularization (40). The rapid proliferation of tumor cells often leads to a lack of oxygen and nutrients in the cells, and new blood supply must be established to meet the increasing metabolic demands (41). When tumors reach a certain volume (typically 1–2 mm in diameter), they will induce the massive expression of angiogenesis-related factors to establish a new vascular network to support their occurrence and development (42). Tumor blood vessels are morphologically different from normal blood vessels. They are abnormally expanded and tortuous, with abnormal branches and disordered networks. There are large gaps among the endothelial cells of these vessels, resulting in high permeability, aggravated vascular leakage, and increased interstitial pressure. Immature tumor neovascularization, missing intraductal support structures, and the rupture or loss of basement membranes can lead to the formation of multiple vascular blind ends and arteriovenous shunt (43). The vascularization at the center of a tumor is lower than that at the peripheral regions, and as a result, the long-existing compression from interstitial internal pressure on the blood vessels reduces blood perfusion and causes tumor ischemia and necrosis (44). The above pathological characteristics of tumor blood vessels result in abnormal blood perfusion and vascular permeability in tumors. However, vessels feeding benign breast lesions (e.g., breast fibroadenoma, intraductal papilloma, and breast adenosis) often originate from the normally growing or expanding vessels inside the glands and thus typically have a normal vascular structure and venous return (45). The number, structure, and morphologies of blood vessels differ significantly between benign and malignant breast lesions. The vascular heterogeneity of benign and malignant breast lesions provides a pathological basis for medical imaging. Specifically, the vascular morphological differences between benign and malignant breast lesions are the optimal diagnostic criterion for distinguishing between the two (46).

At present, one of the most important methods for CUS to evaluate blood flow information in lesions is DUS, which can simultaneously reveal the changes in the structure and morphology of tissues and organs, and the movement of tissues and vessels. DUS can evaluate the blood flow of tumors, thus improving the capability of US diagnostic technology to distinguish between benign and malignant tumors (47). It is mainly composed of color Doppler, spectral Doppler, and energy Doppler. However, its performance in detecting tumor blood flow is limited by many factors, including that: (I) color Doppler is dependent on the angle of incidence of US; (II) it is difficult to display blood vessels with small flow volume and low flow rates; and (III) the aliasing of color signals can easily occur when detecting high-speed blood flow (48-50). For spectral Doppler, the angle of incidence of the ultrasonic beam, the size of the color sampling frames, and the noise threshold of the system all affect the imaging quality, and blood streams with a small flow volume and low flow rate are also difficult to display on spectral Doppler. Energy Doppler cannot show the nature, direction, or velocity of blood flow. Due to its poor ability to resist motion interference, this technique is prone to sparkle artifacts, which can result in poor imaging quality. Some authors have even suggested that CUS can no longer meet the needs of clinical diagnosis and treatment due to its low SE and low SP (51).

As a novel US diagnostic method, CEUS is a pure blood-pool imaging technology (52). During CEUS, the contrast agent SonoVue (with sulfur hexafluoride as the microbubbles, and a surface coated with phospholipid as the stabilizer) was injected into the peripheral veins (53). Next, a series of US-related technologies were used to prolong the life of the microbubbles, highlighting the harmonic components generated by the microbubbles. Further, a series of US-related technologies were used to distribute the bubbles in the tumor microcirculation, thus revealing tumor microvascular perfusion and displaying the whole profile of tumor neovascularization (54). Therefore, it is of great value in the differential diagnosis of benign and malignant breast lesions.

The characteristic manifestations of tumors during CEUS are closely related to the pathophysiology during tumorigenesis and are caused by the structural and morphological abnormalities of the new blood vessels. Most studies have demonstrated that the typical CEUS findings of breast malignant masses include: (I) a hyperenhanced malignant mass, along with radial irregular enhancement in the surrounding areas, and an increased number of blood vessels in the malignant mass. These new tumor vessels have incomplete basement membranes, with a rich anastomoses among them, which can easily form arteriovenous fistulae (55) resulting in the rapid entry and passage of contrast agent in the lesion. As the dose of contrast agent entering the mass in a short time is significantly higher than that of the surrounding normal tissues, hyperenhancement of the lesion is observed. Zhang et al. (56) found that the expression of vascular endothelial growth factor (VEGF) was significantly higher in the peripheral part of the lesion than the central part, and the microvascular density was also significantly higher in the peripheral part than the central part. Thus, the malignant tumor infiltrates to the periphery in a “crab foot-like” fashion, and there is radial enhancement around the lesion during imaging; (II) uneven enhancement in the lesion, accompanied by local filling defects. Balleyguier et al. (57) found the malignant vessels were disordered and unevenly distributed inside the tumors; they were dense in some parts but sparse in others. Meanwhile, the malignant vessels had abnormal morphologies, and their lumens could be either thick or thin. Factors such as infiltrative growth and distant metastasis can easily lead to thrombus formation in malignant vessels, resulting in the narrowing and blockage of the lumen. The uneven distribution of contrast agent leads to uneven enhancement on CEUS. Because the VEGF expression level in the peripheral part of the lesion is higher than that in the central part, the malignant vessels are mainly distributed in the tumor edges, resulting in a relative shortage of blood supply in the central part of the lesion. Accordingly, as the blood supply in the central part of the lesion cannot meet the nutrition requirements for tumor cell growth, ischemia or even necrosis occur in the local areas of the tumor, resulting in the signs of focal filling defects on CEUS (58). (III) the extent to which the lesion is larger than that before CEUS. In the early stage of the infiltration and growth of the mass to the periphery, a large number of new blood vessels are generated around the lesion. However, the morphologies of the lesion do not change during this period and thus no abnormality will be observed on CUS. Conversely, CEUS reveals the entire vascular perfusion process, showing the new blood vessels around the lesion. Compared to the CUS, CEUS more accurately detects the outermost edges of an infiltratively growing tumor. As a result, the extent of the lesions measured on CEUS will be larger than that measured by CUS (59). CEUS enables both qualitative and quantitative analyses. Wan et al. (60) conducted a retrospective analysis of the quantitative parameters of CEUS and found that there were significant differences between malignant and benign breast lesions in the peak intensity, time to peak, and ascending slope; more specifically, the malignant lesions had a higher peak intensity, shorter time to peak, and higher ascending slope than the benign lesions, which might be due to the abnormal neovascular structure and morphologies of the tumors (61). Wang et al. (62) compared more quantitative CEUS parameters in an analysis with a larger sample size and found that in addition to the time to peak and peak intensity AUCs, the time-intensity AUCs also differed significantly between the benign and malignant breast lesions. They also found that the diagnostic performance of the quantitative parameters was inferior to that of the qualitative indicators, and the predictive ability of the quantitative parameters did not yet meet the requirements for routine clinical application. Li et al. (63) and Leng et al. (64) also found that the selection of the ROI had a great effect on the results of the quantitative analysis. Therefore, while the quantitative analysis in CEUS has clinical value in differentiating between benign and malignant breast nodules, its analytical application still needs further research.

In the present study, HiFR-CEUS was performed for the examination of BI-RADS 4 breast nodules, and a qualitative analysis was used to discriminate between benign and malignant lesions. On the basis of CUS, HiFR-CEUS increased the sampling FR, increased the sampling data, and prolonged the visualization of lesion perfusion within a limited time window (the interval between the arterial and venous phases), which enabled the movement trajectory of microbubbles in the blood vessels (i.e., the feature information of lesion perfusion) to be more accurately tracked. With high temporal resolution, HiFR-CEUS can track the movement of microbubbles in the arterial phase, portal venous phase, and delayed phase (or extrahepatic venous phase) in real time, thus enabling the detection and etiologic diagnosis of lesions.

In the current study, HiFR-CEUS misdiagnosed a benign nodule as malignant (which was pathologically confirmed to be serous mastitis). HiFR-CEUS showed that the nodule had uneven hyperenhancement, radial enhancement at the edges, and enlarged enhancement areas (i.e., malignant characteristics); however, these characteristics might have been caused by the metamorphic exudation of inflammatory foci, varying degrees of necrosis, and the presence of rich new blood vessels. For such lesions, a careful history should be taken to rule out tumors before a diagnosis of serous mastitis is made. In patients without obvious medical histories or signs, plasma cell mastitis in the inflammatory phase is difficult to distinguish from breast cancer by either CUS or CEUS. Thus, biopsy is indicated even if the clinical history is typical.

In the present study, 16 malignant nodules were misdiagnosed as benign, including three cases of intraductal carcinoma and 11 cases of invasive ductal carcinoma, all of which were <10 mm in diameter. HiFR-CEUS showed that these lesions had uniform hyperenhancement and clear edges after enhancement (i.e., benign characteristics). However, this might be explained by the following facts: these lesions were small in size; they had not invaded the surrounding glands; no necrotic areas had formed inside the tumors; and the thick blood vessels in the peripheral areas were not obvious. After these malignancies were pathologically confirmed, we reviewed the immunohistochemical results of these misdiagnosed cases and found that their human epidermal growth factor 2 and Ki-67 were negative or very low, indicating that the neoangiogenesis activities of these masses were low, as were the cancer cell proliferation activities (65,66). As a result, there were no obvious changes in the vascular structure, and the lesions were displayed as “benign” on CEUS. Thus, sonographers should pay special attention to small lesions (<10 mm in diameter) that are suspicious on CUS (and diagnosed as BI-RADS 4 nodules) but display benign features on HiFR-CEUS. Patients should be closely followed up within a short period, and consideration should be given to arranging a histological biopsy to confirm the diagnosis. In two cases of mucinous carcinoma, HiFR-CEUS showed uniform hypoenhancement, probably due to the large amount of extracellular mucus inside the tumor, and only a small amount of contrast agent entering the edges of the lesion.

This study had some limitations. First, the use of HiFR-CEUS largely depended on the experience and expertise of the sonographers, and the judgments of “possibly benign” or “possibly malignant” for the same lesion might also be different due to different subjective factors. Second, only the qualitative diagnosis of CEUS was used, and the quantitative diagnosis of CEUS was not used. Third, the sample size was small, and the pathological type was single; thus, studies with larger sample sizes need to be conducted.

Value of AI (US Smart Breast technology) combined with HiFR-CEUS in the differential diagnosis of BI-RADS 4 breast nodules

US Smart Breast technology automatically identifies breast lesions and automatically analyzes the size, shape, edges, internal echo characteristics, and internal blood flow characteristics of a breast nodule for intelligent evaluation to make a diagnosis of “possibly benign” or “possibly malignant” and reduces the effects of subjective factors. HiFR-CEUS accurately displays the blood perfusion characteristics of breast lesions, providing a basis for the differential diagnosis of benign and malignant breast lesions. The combined use of AI and HiFR-CEUS overcame the effects related to the subjective factors of the sonographers, increased the accurate perfusion characteristics of blood flow inside the nodules, and improved the diagnostic efficiency, making the examination results more reliable.

In the combination group of the current study, three patients were found to be misdiagnosed based on a comparison with the pathological results. Two benign nodules (both of which were pathologically confirmed to be inflammatory masses) were misdiagnosed as malignant. In one case, the lesion had a large volume, an irregular shape, angulated edges, uneven internal echoes, and visible blood flow signals in the periphery and inside the tumor, and was diagnosed as malignant by the US Smart Breast technology, as well as by the combination strategy. In another case, HiFR-CEUS showed uneven hyperenhancement, radial enhancement at the edges, and an expanded enhancement range (i.e., malignant characteristics), for which HiFR-CEUS made a diagnosis of “possibly malignant”, and therefore a diagnosis of malignancy was also made using the combination strategy. One malignant nodule (which was pathologically confirmed to be intraductal carcinoma with internal microcalcifications) was misdiagnosed as benign. As a small lesion with clear edges and regular shape, it was diagnosed as benign by Smart Breast. HiFR-CEUS also diagnosed it as benign because the lesion was small, did not invade the surrounding glands, had no internal necrotic areas, and had no obvious thick blood vessels in surrounding areas. CEUS showed uniform hyperenhancement and clear edges (i.e., benign characteristics). Thus, it was misdiagnosed as benign using the combination strategy.

Conclusions

Both AI (US Smart Breast) and HiFR-CEUS are effective tools for the differentiation of benign and malignant BI-RADS 4 breast nodules. Compared with AI alone or HiFR-CEUS alone, the combined use of these two methods had higher diagnostic performance in distinguishing between benign and malignant BI-RADS 4 breast nodules. Thus, the combination of the two methods could further improve diagnostic ACC and guide clinical decision making.

Acknowledgments

None.

Footnote

Reporting Checklist: The authors have completed the STARD reporting checklist. Available at https://gs.amegroups.com/article/view/10.21037/gs-24-187/rc

Data Sharing Statement: Available at https://gs.amegroups.com/article/view/10.21037/gs-24-187/dss

Peer Review File: Available at https://gs.amegroups.com/article/view/10.21037/gs-24-187/prf

Funding: None.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-24-187/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by Ethics Committee of Taizhou People’s Hospital (No. KY202107303) and informed consent was taken from all the patients.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. DeSantis CE, Ma J, Gaudet MM, et al. Breast cancer statistics, 2019. CA Cancer J Clin 2019;69:438-51. [Crossref] [PubMed]
2. Global Burden of Disease Cancer Collaboration. Global, Regional, and National Cancer Incidence, Mortality, Years of Life Lost, Years Lived With Disability, and Disability-Adjusted Life-Years for 29 Cancer Groups, 1990 to 2016: A Systematic Analysis for the Global Burden of Disease Study. JAMA Oncol 2018;4:1553-68. [Crossref] [PubMed]
3. Chen W, Zheng R, Baade PD, et al. Cancer statistics in China, 2015. CA Cancer J Clin 2016;66:115-32. [Crossref] [PubMed]
4. Ginsburg O, Bray F, Coleman MP, et al. The global burden of women's cancers: a grand challenge in global health. Lancet 2017;389:847-60. [Crossref] [PubMed]
5. Liang YC, Jia CM, Xue Y, et al. Diagnostic value of contrast-enhanced ultrasound in breast lesions of BI-RADS 4. Zhonghua Yi Xue Za Zhi 2018;98:1498-502. [Crossref] [PubMed]
6. He P, Cui LG, Chen W, et al. Subcategorization of Ultrasonographic BI-RADS Category 4: Assessment of Diagnostic Accuracy in Diagnosing Breast Lesions and Influence of Clinical Factors on Positive Predictive Value. Ultrasound Med Biol 2019;45:1253-8. [Crossref] [PubMed]
7. Wang Y, Liu G, Ye S, et al. The values of magnetic resonance dynamic enhancement and mammography in the diagnosis of benign and malignant breast lesions. Journal of Imaging Research and Medical Applications 2022;6:119-21.
8. Wang L, Yang W, Xie X, et al. Application of digital mammography-based radiomics in the differentiation of benign and malignant round-like breast tumors and the prediction of molecular subtypes. Gland Surg 2020;9:2005-16. [Crossref] [PubMed]
9. Leithner D, Wengert GJ, Helbich TH, et al. Clinical role of breast MRI now and going forward. Clin Radiol 2018;73:700-14. [Crossref] [PubMed]
10. Jun W, Cong W, Xianxin X, et al. Meta-Analysis of Quantitative Dynamic Contrast-Enhanced MRI for the Assessment of Neoadjuvant Chemotherapy in Breast Cancer. Am Surg 2019;85:645-53.
11. Virostko J, Hainline A, Kang H, et al. Dynamic contrast-enhanced magnetic resonance imaging and diffusion-weighted magnetic resonance imaging for predicting the response of locally advanced breast cancer to neoadjuvant therapy: a meta-analysis. J Med Imaging (Bellingham) 2018;5:011011. [Crossref] [PubMed]
12. Martaindale SR, Breast MR. Imaging: Atlas of Anatomy, Physiology, Pathophysiology, and Breast Imaging Reporting and Data Systems Lexicon. Magn Reson Imaging Clin N Am 2018;26:179-90. [Crossref] [PubMed]
13. Gong W, Zhu J, Hong C, et al. Diagnostic accuracy of cone-beam breast computed tomography and head-to-head comparison of digital mammography, magnetic resonance imaging and cone-beam breast computed tomography for breast cancer: a systematic review and meta-analysis. Gland Surg 2023;12:1360-74. [Crossref] [PubMed]
14. Xiang W, Rao H, Zhou L. A meta-analysis of contrast-enhanced spectral mammography versus MRI in the diagnosis of breast cancer. Thorac Cancer 2020;11:1423-32. [Crossref] [PubMed]
15. Gentili F, Guerrini S, Mazzei FG, et al. Dual energy CT in gland tumors: a comprehensive narrative review and differential diagnosis. Gland Surg 2020;9:2269-82. [Crossref] [PubMed]
16. Liu Y, Kim J, Qu F, et al. CT Features Associated with Epidermal Growth Factor Receptor Mutation Status in Patients with Lung Adenocarcinoma. Radiology 2016;280:271-80. [Crossref] [PubMed]
17. Shao SH, Li CX, Yao MH, et al. Incorporation of contrast-enhanced ultrasound in the differential diagnosis for breast lesions with inconsistent results on mammography and conventional ultrasound. Clin Hemorheol Microcirc 2020;74:463-73. [Crossref] [PubMed]
18. Zhang Z, Zhao QY, Wang Y, et al. Application of New Ultrasound Contrast Agent Sonazoid in the Diagnosis and Treatment of Breast Cancer and Its Sentinel Lymph Node. Chinese Journal of Medical Imaging 2023;31:189-92.
19. Sigrist RMS, Liau J, Kaffas AE, et al. Ultrasound Elastography: Review of Techniques and Clinical Applications. Theranostics 2017;7:1303-29. [Crossref] [PubMed]
20. Menezes R, Sardessai S, Furtado R, et al. Correlation of Strain Elastography with Conventional Sonography and FNAC/Biopsy. J Clin Diagn Res 2016;10:TC05-10. [Crossref] [PubMed]
21. Moon JH, Koh SH, Park SY, et al. Comparison of the SR(max), SR(ave), and color map of strain-elastography in differentiating malignant from benign breast lesions. Acta Radiol 2019;60:28-34. [Crossref] [PubMed]
22. Yoon JH, Song MK, Kim EK. Semi-Quantitative Strain Ratio in the Differential Diagnosis of Breast Masses: Measurements Using One Region-of-Interest. Ultrasound Med Biol 2016;42:1800-6. [Crossref] [PubMed]
23. Dietrich CF, Averkiou M, Nielsen MB, et al. How to perform Contrast-Enhanced Ultrasound (CEUS). Ultrasound Int Open 2018;4:E2-E15. [Crossref] [PubMed]
24. Carriero A, Groenhoff L, Vologina E, et al. Deep Learning in Breast Cancer Imaging: State of the Art and Recent Advancements in Early 2024. Diagnostics (Basel) 2024;14:848. [Crossref] [PubMed]
25. Balkenende L, Teuwen J, Mann RM. Application of Deep Learning in Breast Cancer Imaging. Semin Nucl Med 2022;52:584-96. [Crossref] [PubMed]
26. Aruleba K, Obaido G, Ogbuokiri B, et al. Applications of Computational Methods in Biomedical Breast Cancer Imaging Diagnostics: A Review. J Imaging 2020;6:105. [Crossref] [PubMed]
27. Shen WC, Chang RF, Moon WK, et al. Breast ultrasound computer-aided diagnosis using BI-RADS features. Acad Radiol 2007;14:928-39. [Crossref] [PubMed]
28. Han S, Kang HK, Jeong JY, et al. A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 2017;62:7714-28. [Crossref] [PubMed]
29. Kim K, Song MK, Kim EK, et al. Clinical application of S-Detect to breast masses on ultrasonography: a study evaluating the diagnostic performance and agreement with a dedicated breast radiologist. Ultrasonography 2017;36:3-9. [Crossref] [PubMed]
30. Huang Y, Yao Z, Li L, et al. Deep learning radiopathomics based on preoperative US images and biopsy whole slide images can distinguish between luminal and non-luminal tumors in early-stage breast cancers. EBioMedicine 2023;94:104706. [Crossref] [PubMed]
31. Barr RG. Future of breast elastography. Ultrasonography 2019;38:93-105. [Crossref] [PubMed]
32. Zhao YX, Liu S, Hu YB, et al. Diagnostic and prognostic values of contrast-enhanced ultrasound in breast cancer: a retrospective study. Onco Targets Ther 2017;10:1123-9. [Crossref] [PubMed]
33. Shen RX, Yang LC, Luo XM, et al. Qualitative Features of Contrast-enhanced Ultrasonography in Benign and Malignant Breast Lesions Based on Data of a Multi-center Study in China: A Retrospective Study. Chinese Journal of Medical Imaging 2018;26:885-9.
34. Luo J, Chen JD, Chen Q, et al. Application value of predictive models of contrast-enhanced ultrasound in the evaluation of breast imaging reporting and data system 4 breast lesions. Chinese Journal of Medical Ultrasound 2016;13:459-65. (Electronic Edition).
35. Hosny A, Parmar C, Quackenbush J, et al. Artificial intelligence in radiology. Nat Rev Cancer 2018;18:500-10. [Crossref] [PubMed]
36. Wu GG, Zhou LQ, Xu JW, et al. Artificial intelligence in breast ultrasound. World J Radiol 2019;11:19-26. [Crossref] [PubMed]
37. Park HJ, Kim SM, La Yun B, et al. A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of breast masses on ultrasound: Added value for the inexperienced breast radiologist. Medicine (Baltimore) 2019;98:e14146. [Crossref] [PubMed]
38. Becker AS, Mueller M, Stoffel E, et al. Classification of breast cancer in ultrasound imaging using a generic deep learning analysis software: a pilot study. Br J Radiol 2018;91:20170576. [Crossref] [PubMed]
39. Obenauer S, Hermann KP, Grabbe E. Applications and literature review of the BI-RADS classification. Eur Radiol 2005;15:1027-36. [Crossref] [PubMed]
40. Zhang M, Liu J, Liu G, et al. Anti-vascular endothelial growth factor therapy in breast cancer: Molecular pathway, potential targets, and current treatment strategies. Cancer Lett 2021;520:422-33. [Crossref] [PubMed]
41. Gillies RJ, Brown JS, Anderson ARA, et al. Eco-evolutionary causes and consequences of temporal changes in intratumoural blood flow. Nat Rev Cancer 2018;18:576-85. [Crossref] [PubMed]
42. Craft PS, Harris AL. Clinical prognostic significance of tumour angiogenesis. Ann Oncol 1994;5:305-11. [Crossref] [PubMed]
43. Li AH, Zhou JH. Assessment of angiogenesis and anti-Angiogenesis of tumors by contrast-enhanced ultrasound. Contemporary Medicine 2008;1:145-8.
44. Bian XW, Jiang XF, Chen JH, et al. Increased angiogenic capabilities of endothelial cells from microvessels of malignant human gliomas. Int Immunopharmacol 2006;6:90-9. [Crossref] [PubMed]
45. Wang XY, Lan CY, Wei HM, et al. Contrast-enhanced Ultrasonography in Character and Diagnosis of Breast Fibroadenoma. Chinese Journal of Ultrasound in Medicine 2013;29:316-20.
46. Zeng Q, Li F, Yang W. Application of contrast-enhanced ultrasonography in the diagnosis of breast tumors. Chinese Journal of Oncology Prevention and Treatment 2015;457-9.
47. Evans DH, Jensen JA, Nielsen MB. Ultrasonic colour Doppler imaging. Interface Focus 2011;1:490-502. [Crossref] [PubMed]
48. Hu J, Liu Y, Deng X. Application of ultrasonography for breast tumors. Chinese Journal of Hemorheology 2018;28:361-4.
49. del Cura JL, Elizagaray E, Zabala R, et al. The use of unenhanced Doppler sonography in the evaluation of solid breast lesions. AJR Am J Roentgenol 2005;184:1788-94. [Crossref] [PubMed]
50. Ibrahim R, Rahmat K, Fadzli F, et al. Evaluation of solid breast lesions with power Doppler: value of penetrating vessels as a predictor of malignancy. Singapore Med J 2016;57:634-40. [Crossref] [PubMed]
51. Menezes GL, Knuttel FM, Stehouwer BL, et al. Magnetic resonance imaging in breast cancer: A literature review and future perspectives. World J Clin Oncol 2014;5:61-70. [Crossref] [PubMed]
52. Sheeran PS, Matsunaga TO, Dayton PA. Phase change events of volatile liquid perfluorocarbon contrast agents produce unique acoustic signatures. Phys Med Biol 2014;59:379-401. [Crossref] [PubMed]
53. Le J, Chang C, Chen M, et al. Comparison of conventional ultrasound combined with contrast-enhanced ultrasound and conventional ultrasound in differential diagnosis of breast lesions. Shanghai Medical Imaging 2012;21:182-5.
54. Liu H, Jiang YX, Liu JB, et al. Enhancement pattern of breast lesions in contrast enhanced ultrasound: comparative observation with histopathologic findings. Chinese Journal of Medical Imaging Technology 2009;25:783-5.
55. Liu GQ, Dong BL. The Correlation of Contrast-enhanced Ultrasonography and Microvascular Density in Breast Tumors. Journal of Chinese Oncology 2011;17:184-6.
56. Zhang Y, Jiang Q, Chen J, et al. Correlation of contrast-enhanced ultrasonography with microvessel density and vascular endothelial growth factor expression of beast tumors. Chinese Journal of Ultrasonography 2012;21:52-5.
57. Balleyguier C, Opolon P, Mathieu MC, et al. New potential and applications of contrast-enhanced ultrasound of the breast: Own investigations and review of the literature. Eur J Radiol 2009;69:14-23. [Crossref] [PubMed]
58. Huang HH, Xu XH. Quantitative analysis of contrast-enhanced ultrasonography in breast tumors. Journal of Guangdong Medical College 2011;29:24-6.
59. Wei X, Li Y, Zhu Y, et al. Microvascular of breast lesions evaluated by contrast enhancement ultrasound imaging combined with pathologic basis of angiogenesis. Chinese Journal of Clinicians 2011;5:120-4. (Electronic Edition).
60. Wan C, Du J, Fang H, et al. Evaluation of breast lesions by contrast enhanced ultrasound: qualitative and quantitative analysis. Eur J Radiol 2012;81:e444-50. [Crossref] [PubMed]
61. Zhao H, Xu R, Ouyang Q, et al. Contrast-enhanced ultrasound is helpful in the differentiation of malignant and benign breast lesions. Eur J Radiol 2010;73:288-93. [Crossref] [PubMed]
62. Wang Y, Fan W, Zhao S, et al. Qualitative, quantitative and combination score systems in differential diagnosis of breast lesions by contrast-enhanced ultrasound. Eur J Radiol 2016;85:48-54. [Crossref] [PubMed]
63. Li J, Guo L, Yin L, et al. Can different regions of interest influence the diagnosis of benign and malignant breast lesions using quantitative parameters of contrast-enhanced sonography? Eur J Radiol 2018;108:1-6. [Crossref] [PubMed]
64. Leng X, Huang G, Ma F, et al. Regional Contrast-Enhanced Ultrasonography (CEUS) Characteristics of Breast Cancer and Correlation with Microvessel Density (MVD). Med Sci Monit 2017;23:3428-36. [Crossref] [PubMed]
65. Guan XF, Yu LH, Deng Q, et al. Correlation between CEUS and prognostic factors of breast cancer. Chinese Journal of Ultrasound in Medicine 2016;32:1069-72.
66. Wang YT. Recent advances in the correlation between ultrasonography and molecular immunological index in breast carcinoma. Journal of Clinical Ultrasound in Medicine 2017;19:623-4.

(English Language Editor: L. Huleatt)