Definition of learning curve for thyroidectomy: systematic review on the different approaches

Introduction

The learning curve (LC) concept was first introduced in 1936 by TP Wright, an aeronautical engineer (1). The measure of skills and workforce is easy to define in the industrial world through the analysis of costs, production times and product quality. Since this concept was translated to the full spectrum of medical specialities and procedures, its definition has become more complex and controversial. Moreover, with the advent of minimally invasive techniques, LC concept has become a fundamental “dogma”, with specific and potentially dramatic implications, particularly in all fields of surgery (2). Over the years with the ever-increasing demand for specific skills and excellent results, many studies have been performed on the learning process of individual surgical procedures, from the simplest to the most complex.

Thyroidectomy is one of the most common surgical procedures carried out in general and endocrine surgery units, worldwide. There have been more advances in thyroid surgery since the first successful procedure by Kocher in 1872. To minimize surgical morbidity and neck scarring, minimally invasive thyroidectomy has been developed over the past 20 years. The evolution of endoscopic surgery satisfies the esthetic demand, recovery, and limited trauma in nearly all fields of surgical disciplines, including the treatment of differentiated thyroid cancer (3). In 1996, Gagner et al. (4) reported the first endoscopic neck surgery (parathyroidectomy). The first video-assisted thyroid lobectomy was performed by Hüscher et al. (5) in 1997. Miccoli et al. (6) did invasive video-assisted thyroidectomy for papillary carcinoma successfully in 2001. Soon, endoscopic thyroidectomy was developed into scarless operation by Ikeda et al. (7) and Ohgami et al. (8), by using alternative techniques. Various methods of scarless endoscopic thyroidectomy procedure were subsequently introduced in the following decade.

Robotic thyroidectomy has been introduced to overcome the limitations of endoscopic procedures providing a three-dimensional 10–12-fold magnified view, allowing an easier identification of the parathyroid glands and the recurrent laryngeal nerve (RLN) with a safer and more precise dissection (9). Unlike endoscopic thyroidectomy, robotic approach provides fine motion scaling, hand-tremor filtering, innovative instrumentation with extended freedom of motion, as well as surgical education (9,10). The first robotic application for endoscopic thyroidectomy was performed via a gasless transaxillary approach by Kang and colleagues in South Korea in 2009 (11).

These new exciting technologies are complex and require experienced thyroid surgeons and surgical teams to ensure safe implementation. The aim of this systematic review is to investigate the learning process through the different surgical approaches to the thyroid gland, analysing how the surgeons skill and expertise on each surgical technique has been developed and calculated, thus providing an overview of the parameters used in defining surgical proficiency. We present this article in accordance with the PRISMA reporting checklist (available at https://gs.amegroups.com/article/view/10.21037/gs-22-730/rc) (12).

Methods

The study was carried out according to current ethical standards. The study protocol has been registered on researchregistry.com database (reviewregistry1425).

Search strategy

Searches were conducted for all English language full-text articles published until June 2022. The following database sources were searched: PubMed (MEDLINE), Scopus, Cochrane Library, EMBASE, Web of Science. The following term combination was used: ((thyroidectomy) OR (thyroid surgery)) AND (learning curve); separately we analyzed the following term combinations (endoscopic thyroidectomy) AND (learning curve); (robotic thyroidectomy) AND (learning curve); (minimally-invasive thyroidectomy) AND (learning curve); ((conventional thyroidectomy) OR (open thyroidectomy)) AND (learning curve). Records were screened for relevance based on their title and abstract and successively the full text of the remaining articles was analyzed. Furthermore, the references list of each selected article was analyzed to identify additional relevant studies.

We appraised the risk of bias of included randomized controlled trials (RCTs) according to the Cochrane risk of bias assessment tool (13). This tool evaluates the following domains: (I) random sequence generation, (II) allocation concealment, (III) blinding of participants and personnel, (IV) blinding of outcome assessment, (V) incomplete outcome data, (VI) selective outcome reporting, and (VII) other potential sources of bias. Two co-authors (AP and AI) performed the risk of bias assessment independently and disagreements were rectified by consensus and consultation with a third co-author (AG).

Inclusion criteria

The types of studies eligible for inclusion were only original articles (retrospective, prospective, randomized clinical trials). Criteria of inclusion of potential studies in this review were cohort studies or case series reporting almost 30 procedures that investigated the learning process to thyroidectomy (including hemi- and total thyroidectomy) describing clearly the minimum number of cases required and the main evaluation items used to establish it regardless of the number of surgeons involved in each study. Conventional, endoscopic and robotic approaches were included in the research and separately investigated. Only English-language studies were considered.

Endpoint

The endpoint of this systematic review is to define the concept of LC applied to thyroid gland surgery through the different surgical approaches by assessing the number of procedures required for a single surgeon to achieve competence and establishing a method as objective as possible to evaluate it.

Data extraction and synthesis

Two authors (MT and DC) independently screened each record from full text articles for eligibility and extracted the data, including quality analysis. Disagreement was resolved by discussion and consensus; if no agreement was reached, a third author was consulted (MB).

Descriptive statistics were produced from the dataset: continuous data were pooled and are reported as percentages. There was no comparative statistical analysis.

Results

A total of 161, 226 and 120 relevant references concerning the assessment of LC respectively for robotic, endoscopic and conventional thyroidectomy were found, and after screening a total 45 full-text articles were assessed for eligibility (Figures 1-3). The main characteristics of the included studies are summarized in Tables 1-6 and represented graphically in Figure 4.

Figure 1 PRISMA flowchart—robotic thyroidectomy. LC, learning curve.

Figure 2 PRISMA flowchart—endoscopic thyroidectomy. LC, learning curve; MIVAT, mini-invasive video assisted thyroidectomy.

Figure 3 PRISMA flowchart—conventional thyroidectomy. LC, learning curve.

Figure 4 Main characteristics of the included studies. All values are expressed as percentage (%). CT, conventional thyroidectomy; RT, robotic thyroidectomy; ET, endoscopic thyroidectomy; CUSUM, cumulative sum; IONM, intraoperative neuromonitoring.

Robotic thyroidectomy

After screening the abstracts and full-texts, a total of sixteen articles were included (14-29) (Figure 1). The techniques investigated in the studies selected were: BABA (bilateral axillo-breast approach) (6 reports) (14-19), transaxillary (7 reports) (20-26), transoral (2 reports) (27,28), retroauricular (1 report) (29).

The main intra- and postoperative complications including bleeding, hypocalcemia and RLN injuries (transient or permanent) were compared before and after the LC in seven studies (15,16,21-23,25,29). The other trials considered the overall postoperative complication rate. Two studies (18,23) reported the use of IONM (intraoperative neuromonitoring) during surgery.

BABA

All studies are retrospective. Series ranged from a minimum of 100 cases to a maximum of 317 cases with a total of 1,243 patients. Four authors used the moving average method to investigate the LC (14,15,17,18), only in two papers the CUSUM (cumulative sum) method was adopted (16,19). The median number of operations required to achieve competence was established to be of 38 (min–max: 30–50) procedures for a single surgeon. All reports used operative time to assess surgical proficiency. Sun et al. (17) in a large series identified two peaks: an early proficiency at about 35 cases and a late proficiency gained after 80 cases. Two studies compared the rate of major postoperative complications before and after LC achievement.

Transaxillary

This group includes seven papers, six of which analyze the LC process applied to the gasless technique (20-25) and one to the single-port technique (26). Three studies are prospective while the remaining four are retrospective. In four papers the experience of a single surgeon was analyzed. All authors utilized the OT (operative time) to evaluate the achievement of competence. Only one author (21) considered other variables in parallel, i.e., the intraoperative blood loss, the length of stay, the number of lymph nodes retrieved, and the degree of complete resection. Series ranged from a minimum of 31 cases to a maximum of 1,769 cases for a total of 3,057 patients. Of the total number of procedures performed, 1,651 were hemithyroidectomies (54%). The analysis methods used included CUSUM for one report only (26), moving average curve in four studies while two authors proposed a comparative analysis between groups with different levels of experience (22,23). The median number of surgical procedures required to complete the learning curve resulted in 37 surgeries (min–max: 10–50).

Transoral

In our research only two studies (27,28) evaluate the LC applied to the TORT (transoral robotic thyroidectomy). Both are retrospective, one uses the CUSUM (28), another the moving average method (27). Kim et al. (28) in addition to the OT, considered the complication rate, and the surgical success (procedure conversion) rate. A total of 228 procedures (81.6% of which hemithyroidectomies) were analyzed and the median number of operations needed to complete the LC was established to be 40.

Retroauricular

Only the recent study proposed by Han et al. (29) investigated LC for robotic thyroidectomy with retroauricular approach. The study retrospectively evaluated the experience of a single surgeon for a total of 36 hemithyroidectomies. By applying the CUSUM method on the OT, Han et al. established the achievement of competence starting from 15 procedures.

Endoscopic thyroidectomy

After screening a total of 28 studies were selected: 22 (30-51) for thyroidectomy by remote access and 6 (52-57) for MIVAT (Figure 2).

The techniques investigated for remote-access thyroidectomy in the selected studies were: transoral endoscopic thyroidectomy trans-vestibular approach) (9 reports (TOETVA) (30-38), breast approach (5 reports) (39-43), transaxillary (3 reports) (44-46), BABA (2 reports) (47,48), subclavian (2 reports) (49,50), retroauricular (1 report) (51). Bleeding, hypocalcemia and RNL injuries were compared before and after proficiency achievement in eight papers (36,40-43,45,47,50) with other trials considering the overall postoperative complication rate. Nine authors reported the use of IONM during surgery (30,32,33,36-38,48,50,51).

TOETVA

Nine studies in the current literature concern the LC applied to the TOETVA (30-38). Six are retrospective and three prospective. The total number of procedures amounts to 1,146 (75% of them hemithyroidectomies). The experience of a single surgeon was reported in three papers. Five studies used the CUSUM method. The study proposed by Fernandez-Ranvier (38) considered exclusively the complication rate and severity in LC assessment. The other authors analyzed the OT and in two cases other parameters such as blood loss. The median number of procedures required to achieve proficiency for the TOETVA was thus established at 20 surgeries (min–max: 11–69).

Breast approach

Five of the studies selected in our review belong to this group (39-43). A paper analyzed the LC for robotic thyroidectomy via breast with single-port technique (42). All authors reported the experience of a single surgeon and three of them used the CUSUM analysis (41-43). Series ranged from a minimum of 45 cases to a maximum of 300 cases with a total of 654 patients. 92.2% of the procedures were hemithyroidectomies. The OT was considered the main evaluation item even if other parameters such as complication rate, blood loss, length of stay, lymph nodes removed and parathyroids removal were included in two reports. The median number of surgeries for proficiency was established to be of 27 procedures for a single surgeon (min–max: 22–60). Moreover, the two largest series (39,41) identified a late proficiency after 67 and 150 operations, respectively.

Transaxillary approach

Of the three papers included in this group (44-46), one specifically evaluated the gasless technique (44) and another the single-port technique (45). Only Kwak et al. (44) analyzed LC separately for hemithyroidectomy and total thyroidectomy. In the other two studies, all procedures are hemithyroidectomies. CUSUM is the analysis method used in all reports. Two of them associated different parameters such as postoperative complications, tumor size and lymph nodes removed. The median number of procedures required to gain experience in hemithyroidectomy with an endoscopic transaxillary approach was set at 35 surgeries.

Bilateral-Axillary-Breast-Approach

Two papers (47,48) included in this review retrospectively analyzed the LC for the BABA with endoscopic technique. The two studies used different analysis methods: CUSUM in one case (47), moving average curve (48) in the other. The series were almost similar in number. However, in the study of Wang et al. (48) all procedures were hemithyroidectomy and the author identified an early proficiency at 30 operations, equal to the number established by Liang et al. (47), and a late proficiency after 60 surgeries.

Subclavian

Two retrospective studies in the current literature evaluate LC for endoscopic thyroidectomy with a subclavian approach (49,50). Neither author used the CUSUM and in both papers proficiency was established at 30 procedures for a single surgeon. In both reports, in addition to the OT, complications and blood loss were evaluated.

Retroauricular

Only von Ahnen (51) reporting a series of 150 hemithyroidectomies and using CUSUM analysis and OT as main evaluation item, established the number of procedures needed to complete LC in 53 surgeries for the retroauricular robotic approach.

MIVAT

Six studies (52-57) were included (Figure 2). Five of them were retrospective. Series ranged from a minimum of 36 cases to a maximum of 843 cases with a total of 1,151 patients (91.4% of them were women). Altogether 805 hemithyroidectomies (69.9%) and 346 total thyroidectomies (30.1%) were performed. Two studies (52,56) analyzed exclusively total thyroidectomy in their assessment of LC. In addition to the operative time that was considered in all reports, four papers (52,53,56,57) evaluated other variables including: postoperative complications and postoperative pain, cosmetic results, intraoperative identification of RLN and parathyroid glands, conversion rate, and length of stay. CUSUM analysis was not used in any study. The median number of operations required to achieve competence was established to be of 31 procedures for a single surgeon. Only Del Rio (52) compared bleeding, hypocalcemia and RNL injuries before and after proficiency achievement. Two studies reported the use of IONM during surgery (53,54).

Conventional thyroidectomy

Only one study (58) was included (Figure 3) that compared data obtained from senior experienced surgeons and surgery residents in two different academic hospitals, in terms of OT and complication rates, in order to define the correct shape of the resident’s LC. The surgeries all consisted of total thyroidectomies for a total of 390 patients (64.1% of them women). The CUSUM method was used to evaluate the learning process and proficiency was established at 25–30 procedures with OT becoming similar between residents and experienced surgeons and no significant differences regarding postoperative complications.

Discussion

Assessing a clinician’s performance is challenging. Measures of learning surgical technique fall into two categories: surgical process and patient outcome. A century ago, Theodor Kocher, the father of thyroid surgery, reported that, as his surgical experience increased from 100 to 5,000 thyroidectomies, his patients’ mortality rate decreased from 12.8% to 0.5% (59). The relationship between surgeon experience and technical competence is controversial and a number of different methods for objective assessment of surgical skills have been developed (60,61). Furthermore, new techniques, such as minimally-invasive surgery, require new skills with different LCs (62). In this regard, it is important to highlight how the LC of conventional thyroidectomy is poorly investigated in current literature. Conversely the steps to achieve proficiency in endoscopic and robotic procedures, even if more recently introduced, are widely studied.

However, our research suggests that there are several confounding factors that can generate a bias in the definition of the learning process for thyroidectomy making it impossible to elaborate a meta-analysis. These factors are essentially related to (I) the analysis criteria of the single studies, (II) the surgeon, (III) the procedure.

Analysis criteria

What criterion is used to define a surgeon more or less expert in a given procedure? This concept has not been universally defined. Almost all studies use operative time to assess the LC. However, although operative time may be easily measured and compared, it is not necessarily the most appropriate marker of surgical proficiency (63). Indeed a total of 19 studies considered other variables besides surgical time that seem to be equally or more important for patients’ care (Figure 5): complication rate (12 reports), blood loss (8 reports), conversion rate (4 reports), LOS (5 reports), n. LN removed (3 reports), tumor size (2 reports), degree of complete resection (1 report), cosmetic results (2 reports), intraoperative identification of RLN and PTG (1 report), removal of PTG (1 report), postoperative drainage amount (1 report), and postoperative pain (2 reports). Only the study proposed by Fernandez-Ranvier et al. (38) used complication rate and severity without considering surgical time to determine the LC for TOETVA.

Figure 5 Summary of the main evaluation items (definition and frequency) used in all studies that analyzed the learning curve for thyroid surgery. OT, operative time; IBL, intraoperative blood loss; LOS, length of stay.

The concept of surgical time becomes even more ambiguous when applied to robotic surgery. In this field, it may be subdivided into specific components such as initial robotic system setup, trocar insertion, docking, and console time according to the steps of procedures (63,64). There is a lack of universal agreement on which operative time component is the most relevant for the learning process (64).

Another important variable is the size of the case series. We only considered papers with series greater than 30 cases eligible for research. Therefore, the series analyzed range from a minimum of 30 cases to a maximum of 1,769 cases. Results highlighted that in the larger series the minimum number of cases to be performed for single surgeons is generally higher than in smaller series and above all it is possible to identify two peaks corresponding to an early and a late LC.

Finally, a large heterogeneity also derives from the statistical methodologies used in the various analyses. Only fifteen studies used the CUSUM method to evaluate the LC. The CUSUM chart was adopted by the medical profession in the 1970s to analyze the LC for surgical procedures (65). CUSUM is the running total of differences between the individual data points and the mean of all data points. It makes possible rapid and powerful assessments of changes in means, or in the slopes of trends, in data collected at regular intervals of time (65,66). Thus, CUSUM can be performed recursively and enables investigators to visualize the data for trends not discernable with other approaches (41).

Surgeon

A total of twenty-eight studies established the LC based on the experience of a single surgeon. Individual learning processes may differ from that of surgeons with different training backgrounds in other institutes. Other important variables that overlap with this are background and familiarity with a particular procedure and this is evident among minimally invasive techniques. Endoscopic/robotic thyroidectomy is usually approached when the surgeon has already gained anatomical and technical skills in other procedures. Conversely open thyroidectomy represents the first step of surgical teaching and the learning process starts already during the training period involving more young trainees. Only in a few studies the LC was assessed for residents or surgeons who had just completed the training program. In most cases they were experienced surgeons. Endoscopic surgery requires acquisition of new anatomical perspectives, hand-to-eye coordination and lacks both tactile feedback, and three dimensional vision. The shift from open to minimally invasive approach represents a completely new experience for surgeons and this might take a longer LC for endoscopic thyroidectomy. The surgeon who begins robotic surgery generally has already passed all these steps and moreover draws many advantages from the robotic system, including the precision and accuracy of anatomical dissection with 3D vision, wristed instrumentation with seven degrees of freedom of motion, lack of tremor, and comfortable seated position.

Procedure

The LC for minimally-invasive thyroidectomy might differ depending on the approach (Figures 6,7). The alternative approaches to the thyroid gland can be divided into cervical minimally invasive, extracervical endoscopic (robot-assisted) and transoral operations (NOTES). Indeed, according to the use of CO₂ gas insufflation and the site of incision various remote-access thyroidectomy methods via axillary, breast, anterior chest, postauricular, and transoral routes have been developed to hide neck scarring. MIVAT through a minimal access cervical incision was first introduced by Miccoli (6,67) and it is characterized by a single incision of 1.5–2 cm above the sternal notch. TOET is the only technique that enables thyroid surgery while completely avoiding a cutaneous incision with better cosmetic results. The other techniques still need cutaneous incisions with substantial dissection through planes that are less familiar to the thyroid surgeon, and require staged procedures or bilateral incisions to complete a total thyroidectomy (68). Although various techniques for TOET are described, the most used is the TOETVA, first reported by Anuwong et al. (69,70), due to its surgical outcomes and low complication rate. TOETVA and MIVAT allow for optimal bilateral visualization of anterior neck structures through familiar subplatysmal planes, and a two-sided procedure can be safely performed. The four common robotic approaches are the gasless transaxillary approach, the BABA, the gasless postauricular facelift approach, and the transoral approach (9).

Figure 6 Summary of the different approaches used for endoscopic thyroidectomy in all included studies. MIVAT, mini-invasive video assisted thyroidectomy; TOETVA, transoral endoscopic thyroidectomy trans-vestibular approach; BABA, bilateral axillary-breast approach.

Figure 7 Summary of the different approaches used for robotic thyroidectomy in all included studies. BABA, bilateral axillary-breast approach.

When we analyze the LC related to these procedures we have to consider some technical aspects. First, the midline method is easier due to a similar operative view to the conventional open thyroidectomy (14,41). In contrast, the lateral approaches, such as transaxillary or retroauricular, might require additional time and more operations for the surgeon to familiarize with the anatomy and procedure, especially during contralateral side dissection in cases of total thyroidectomy (41,71). For this reason, it would be appropriate to distinguish between hemi- and total thyroidectomies since there is a very different difficulty level between these procedures when minimally invasive techniques with lateral approach are performed. Moreover, endoscopic and robotic surgeries have a higher complication rate during the learning process (68) and above all they add unusual types of complications not seen with conventional thyroidectomy such as lower lip hypoesthesia and weakness due to mental nerve injury and dissection of the chin area in the transoral approach, numbness of the chest wall, CO₂ embolism, perforation of the neck, chyle leakage, Horner’s syndrome, and burn and trauma of the skin flap. Aesthetic results become a relatively new quality criterion, not evaluated in the conventional approach: Del Rio and Capponi (52,56) considered cosmetic outcomes as an additional parameter to evaluate the LC for MIVAT.

Furthermore, in our research we highlighted that in a total of twelve reports surgeons routinely used neuromonitoring of the RNL. IONM was introduced in 1966 by Shedd as a complement to visualization of the RLN allowing its evaluation and prediction of function (72). In 2011, the first conference of the Polish Research Group for Neuromonitoring of the Polish Club of Endocrine Surgeons was held and neuromonitoring of the RLN and the external branch of the superior laryngeal nerve was standardized (73). Since then, the use of this technique has become increasingly widespread and the number of centers performing operations with IONM continues to grow (74,75). Numerous publications show a high rate of RLN identification with IONM and several studies investigate the LC for IONM (76-82). However, no study focuses on how much the IONM use affects the surgeon’s LC.

Limitations and strengths

Based on our knowledge this is the first study in the literature to provide an overview of LC applied to thyroid surgery through its different approaches. However, this systematic review has some limitations due to the heterogeneity of the methodologies used in the included studies that did not allow us to perform a comparative analysis or to draw objective results. In addition, the nature of the included retrospective studies may lead to publication biases and could distort the conclusions of this review.

Conclusions

Our research has shown how the current literature lacks an objective and universal definition of the LC concept considering both surgical process and patients’ care. The heterogeneity of the analysis methodologies and the quality criteria evaluated, the various surgical techniques and training background of the individual surgeons, are all factors that make it impossible to draw univocal results. This is even more evident when applied to thyroid gland surgery, one of the most performed procedures worldwide, which has evolved in recent years with alternative approaches responding to new needs, new quality standards, and new technologies. Indications and complexities change according to the technique adopted and consequently the proficiency level required is different. Future studies should consider confounding factors and establish parameters that should be consensually recognized in the assessment of surgical performances and skills.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at https://gs.amegroups.com/article/view/10.21037/gs-22-730/rc

Peer Review File: Available at https://gs.amegroups.com/article/view/10.21037/gs-22-730/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://gs.amegroups.com/article/view/10.21037/gs-22-730/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Wright TP. Factors affecting the cost of airplanes. Journal of Aeronautical Science 1936;3:122-8. [Crossref]
2. Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J 2007;83:777-9. [Crossref] [PubMed]
3. Yang Y, Gu X, Wang X, et al. Endoscopic thyroidectomy for differentiated thyroid cancer. ScientificWorldJournal 2012;2012:456807. [Crossref] [PubMed]
4. Gagner M. Endoscopic subtotal parathyroidectomy in patients with primary hyperparathyroidism. Br J Surg 1996;83:875. [Crossref] [PubMed]
5. Hüscher CS, Chiodini S, Napolitano C, et al. Endoscopic right thyroid lobectomy. Surg Endosc 1997;11:877. [Crossref] [PubMed]
6. Miccoli P, Berti P, Bendinelli C, et al. Minimally invasive video-assisted surgery of the thyroid: a preliminary report. Langenbecks Arch Surg 2000;385:261-4. [Crossref] [PubMed]
7. Ikeda Y, Takami H, Sasaki Y, et al. Endoscopic neck surgery by the axillary approach. J Am Coll Surg 2000;191:336-40. [Crossref] [PubMed]
8. Ohgami M, Ishii S, Arisawa Y, et al. Scarless endoscopic thyroidectomy: breast approach for better cosmesis. Surg Laparosc Endosc Percutan Tech 2000;10:1-4. [Crossref] [PubMed]
9. Tae K. Robotic thyroid surgery. Auris Nasus Larynx 2021;48:331-8. [Crossref] [PubMed]
10. Tae K, Ji YB, Song CM, et al. Robotic and Endoscopic Thyroid Surgery: Evolution and Advances. Clin Exp Otorhinolaryngol 2019;12:1-11. [Crossref] [PubMed]
11. Kang SW, Jeong JJ, Nam KH, et al. Robot-assisted endoscopic thyroidectomy for thyroid malignancies using a gasless transaxillary approach. J Am Coll Surg 2009;209:e1-7. [Crossref] [PubMed]
12. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009;339:b2700. [Crossref] [PubMed]
13. Higgins JP, Altman DG, Gøtzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. [Crossref] [PubMed]
14. Kim WW, Jung JH, Park HY. A single surgeon's experience and surgical outcomes of 300 robotic thyroid surgeries using a bilateral axillo-breast approach. J Surg Oncol 2015;111:135-40. [Crossref] [PubMed]
15. Kim WW, Jung JH, Park HY. The Learning Curve for Robotic Thyroidectomy Using a Bilateral Axillo-Breast Approach From the 100 Cases. Surg Laparosc Endosc Percutan Tech 2015;25:412-6. [Crossref] [PubMed]
16. Kim H, Kwon H, Lim W, et al. Quantitative Assessment of the Learning Curve for Robotic Thyroid Surgery. J Clin Med 2019;8:402. [Crossref] [PubMed]
17. Sun HX, Gao HJ, Ying XY, et al. Robotic thyroidectomy via bilateral axillo-breast approach: Experience and learning curve through initial 220 cases. Asian J Surg 2020;43:482-7. [Crossref] [PubMed]
18. You JY, Kim HK, Kim HY, et al. Bilateral axillo-breast approach robotic thyroidectomy: review of a single surgeon's consecutive 317 cases. Gland Surg 2021;10:1962-70. [Crossref] [PubMed]
19. Ouyang H, Xue W, Zhang Z, et al. Learning curve for robotic thyroidectomy using BABA: CUSUM analysis of a single surgeon's experience. Front Endocrinol (Lausanne) 2022;13:942973. [Crossref] [PubMed]
20. Kang SW, Jeong JJ, Yun JS, et al. Robot-assisted endoscopic surgery for thyroid cancer: experience with the first 100 patients. Surg Endosc 2009;23:2399-406. [Crossref] [PubMed]
21. Lee J, Yun JH, Nam KH, et al. The learning curve for robotic thyroidectomy: a multicenter study. Ann Surg Oncol 2011;18:226-32. [Crossref] [PubMed]
22. Kuppersmith RB, Holsinger FC. Robotic thyroid surgery: an initial experience with North American patients. Laryngoscope 2011;121:521-6. [Crossref] [PubMed]
23. Kandil EH, Noureldine SI, Yao L, et al. Robotic transaxillary thyroidectomy: an examination of the first one hundred cases. J Am Coll Surg 2012;214:558-64; discussion 564-6. [Crossref] [PubMed]
24. Lee J, Yun JH, Choi UJ, et al. Robotic versus Endoscopic Thyroidectomy for Thyroid Cancers: A Multi-Institutional Analysis of Early Postoperative Outcomes and Surgical Learning Curves. J Oncol 2012;2012:734541. [Crossref] [PubMed]
25. Park JH, Lee J, Hakim NA, et al. Robotic thyroidectomy learning curve for beginning surgeons with little or no experience of endoscopic surgery. Head Neck 2015;37:1705-11. [Crossref] [PubMed]
26. Park J, Kang LK, Kim K, et al. The learning curve for single-port transaxillary robotic thyroidectomy (SP-TART): experience through initial 50 cases of lobectomy. Updates Surg 2023;75:691-700. [Crossref] [PubMed]
27. Chen YH, Kim HY, Anuwong A, et al. Transoral robotic thyroidectomy versus transoral endoscopic thyroidectomy: a propensity-score-matched analysis of surgical outcomes. Surg Endosc 2021;35:6179-89. [Crossref] [PubMed]
28. Kim KH, Ji YB, Song CM, et al. Learning curve of transoral robotic thyroidectomy. Surg Endosc 2023;37:535-43. [Crossref] [PubMed]
29. Han SH, Ji JY, Cha W, et al. Cumulative sum analysis of the learning curve for robotic retroauricular thyroidectomy. Gland Surg 2023;12:30-8. [Crossref] [PubMed]
30. Razavi CR, Vasiliou E, Tufano RP, et al. Learning Curve for Transoral Endoscopic Thyroid Lobectomy. Otolaryngol Head Neck Surg 2018;159:625-9. [Crossref] [PubMed]
31. Qu R, Wang J, Li J, et al. The Learning Curve for Surgeons Regarding Endoscopic Thyroidectomy via the Oral-vestibular Approach. Surg Laparosc Endosc Percutan Tech 2018;28:380-4. [Crossref] [PubMed]
32. Lira RB, Ramos AT, Nogueira RMR, et al. Transoral thyroidectomy (TOETVA): Complications, surgical time and learning curve. Oral Oncol 2020;110:104871. [Crossref] [PubMed]
33. Luo JH, Xiang C, Wang P, et al. The Learning Curve for Transoral Endoscopic Thyroid Surgery: A Single Surgeon's 204 Case Experience. J Laparoendosc Adv Surg Tech A 2020;30:163-9. [Crossref] [PubMed]
34. Kandil E, Akkera M, Shalaby H, et al. A Single Surgeon's 10-Year Experience in Remote-Access Thyroid and Parathyroid Surgery. Am Surg 2021;87:638-44. [Crossref] [PubMed]
35. Moreno Llorente P, Pascua-Solé M, García Barrasa A, et al. Transoral endoscopic thyroidectomy vestibular approach: Results after 53 first cases. Cir Esp 2023;101:35-42. (Engl Ed). [Crossref] [PubMed]
36. Kuo TC, Duh QY, Wang YC, et al. Practice Patterns and Learning Curve in Transoral Endoscopic Thyroidectomy Vestibular Approach With Neuromonitoring. Front Endocrinol (Lausanne) 2021;12:744359. [Crossref] [PubMed]
37. Chai YJ, Chae S, Oh MY, et al. Transoral Endoscopic Thyroidectomy Vestibular Approach (TOETVA): Surgical Outcomes and Learning Curve. J Clin Med 2021;10:863. [Crossref] [PubMed]
38. Fernandez-Ranvier G, Lieberman B, Guevara D, et al. Transoral Endoscopic Thyroidectomy Vestibular Approach (TOETVA) learning curve: a regression analysis of complication rates and severity. Surg Endosc 2022;36:4839-44. [Crossref] [PubMed]
39. Liu S, Qiu M, Jiang DZ, et al. The learning curve for endoscopic thyroidectomy: a single surgeon's experience. Surg Endosc 2009;23:1802-6. [Crossref] [PubMed]
40. Cao F, Jin K, Cui B, et al. Learning curve for endoscopic thyroidectomy: a single teaching hospital study. Onco Targets Ther 2013;6:47-52. [PubMed]
41. Liao HJ, Dong C, Kong FJ, et al. The CUSUM analysis of the learning curve for endoscopic thyroidectomy by the breast approach. Surg Innov 2014;21:221-8. [Crossref] [PubMed]
42. Zhu G, Zhang X, Tang Z, et al. The Learning Curve of Transareola Single-site Laparoendoscopic Thyroidectomy: CUSUM Analysis of a Single Surgeon's Experience. Surg Laparosc Endosc Percutan Tech 2016;26:364-7. [Crossref] [PubMed]
43. Yu J, Rao S, Lin Z, et al. The learning curve of endoscopic thyroid surgery for papillary thyroid microcarcinoma: CUSUM analysis of a single surgeon's experience. Surg Endosc 2019;33:1284-9. [Crossref] [PubMed]
44. Kwak HY, Kim SH, Chae BJ, et al. Learning curve for gasless endoscopic thyroidectomy using the trans-axillary approach: CUSUM analysis of a single surgeon's experience. Int J Surg 2014;12:1273-7. [Crossref] [PubMed]
45. Cho J, Lee D, Baek J, et al. Single-incision endoscopic thyroidectomy by the axillary approach with gas inflation for the benign thyroid tumor: retrospective analysis for a single surgeon's experience. Surg Endosc 2017;31:437-44. [Crossref] [PubMed]
46. Jasaitis K, Skimelyte M, Maleckas A, et al. Transaxillary gasless endoscopic hemithyroidectomy versus conventional open hemithyroidectomy: early single-centre experience. Updates Surg 2022;74:917-25. [Crossref] [PubMed]
47. Liang TJ, Tsai CY, Liu SI, et al. Multidimensional Analyses of the Learning Curve of Endoscopic Thyroidectomy. World J Surg 2021;45:1446-56. [Crossref] [PubMed]
48. Wang MF, Xia H, Zhao WJ, et al. The Learning Curve and Importance of Collaboration in Endoscopic Thyroidectomy Via Breast Areola Approach: A Single Surgical Team's Experience of 100 Patients. J Craniofac Surg 2022;33:e802-6. [Crossref] [PubMed]
49. Shimizu K, Shimizu K, Okamura R, et al. Video-assisted neck surgery (VANS) using a gasless lifting procedure for thyroid and parathyroid diseases: "The VANS method from A to Z". Surg Today 2020;50:1126-37. [Crossref] [PubMed]
50. Nagaoka R, Sugitani I, Kazusaka H, et al. Learning Curve for Endoscopic Thyroidectomy Using Video-Assisted Neck Surgery: Retrospective Analysis of a Surgeon's Experience with 100 Patients. J Nippon Med Sch 2022;89:277-86. [Crossref] [PubMed]
51. von Ahnen T, Wirth U, von Ahnen M, et al. Endoscopic cephalic access thyroid surgery (EndoCATS) using the retroauricular approach - a single centre retrospective data analysis. Surg Endosc 2022;36:117-25. [Crossref] [PubMed]
52. Del Rio P, Sommaruga L, Cataldo S, et al. Minimally invasive video-assisted thyroidectomy: the learning curve. Eur Surg Res 2008;41:33-6. [Crossref] [PubMed]
53. Dionigi G, Boni L, Rovera F, et al. Defining the learning curve for video-assisted thyroidectomy. Int J Surg 2008;6:S1-3. [Crossref] [PubMed]
54. Samy AK, Ridgway D, Orabi A, et al. Minimally invasive, video-assisted thyroidectomy: first experience from the United Kingdom. Ann R Coll Surg Engl 2010;92:379-84. [Crossref] [PubMed]
55. Lee J, Lee JH, Nah KY, et al. Comparison of endoscopic and robotic thyroidectomy. Ann Surg Oncol 2011;18:1439-46. [Crossref] [PubMed]
56. Capponi MG, Bellotti C, Lotti M, et al. Minimally invasive video-assisted thyroidectomy: Ascending the learning curve. J Minim Access Surg 2015;11:119-22. [Crossref] [PubMed]
57. Pons Y, Vérillaud B, Blancal JP, et al. Minimally invasive video-assisted thyroidectomy: Learning curve in terms of mean operative time and conversion and complication rates. Head Neck 2013;35:1078-82. [Crossref] [PubMed]
58. Tarallo M, Crocetti D, Gurrado A, et al. Achieving the learning curve in total thyroidectomy: a prospective evaluation on resident's training by CUSUM and KPSS analysis. Ann R Coll Surg Engl 2022;104:414-20. [Crossref] [PubMed]
59. Becker WF. Presidential address: Pioneers in thyroid surgery. Ann Surg 1977;185:493-504. [Crossref] [PubMed]
60. Brown C, Abdelrahman T, Patel N, et al. Operative learning curve trajectory in a cohort of surgical trainees. Br J Surg 2017;104:1405-11. [Crossref] [PubMed]
61. Merola G, Cavallaro G, Iorio O, et al. Learning curve in open inguinal hernia repair: a quality improvement multicentre study about Lichtenstein technique. Hernia 2020;24:651-9. [Crossref] [PubMed]
62. Bracale U, Merola G, Sciuto A, et al. Achieving the Learning Curve in Laparoscopic Inguinal Hernia Repair by Tapp: A Quality Improvement Study. J Invest Surg 2019;32:738-45. [Crossref] [PubMed]
63. Kassite I, Bejan-Angoulvant T, Lardy H, et al. A systematic review of the learning curve in robotic surgery: range and heterogeneity. Surg Endosc 2019;33:353-65. [Crossref] [PubMed]
64. Kaul S, Shah NL, Menon M. Learning curve using robotic surgery. Curr Urol Rep 2006;7:125-9. [Crossref] [PubMed]
65. Chaput de Saintonge DM, Vere DW. Why don't doctors use cusums? Lancet 1974;1:120-1. [Crossref] [PubMed]
66. Wohl H. The cusum plot: its utility in the analysis of clinical data. N Engl J Med 1977;296:1044-5. [Crossref] [PubMed]
67. Miccoli P, Matteucci V. Video-assisted surgery for thyroid cancer patients. Gland Surg 2015;4:365-7. [PubMed]
68. Duek I, Duek OS, Fliss DM. Minimally Invasive Approaches for Thyroid Surgery-Pitfalls and Promises. Curr Oncol Rep 2020;22:77. [Crossref] [PubMed]
69. Anuwong A. Transoral Endoscopic Thyroidectomy Vestibular Approach: A Series of the First 60 Human Cases. World J Surg 2016;40:491-7. [Crossref] [PubMed]
70. Anuwong A, Sasanakietkul T, Jitpratoom P, et al. Transoral endoscopic thyroidectomy vestibular approach (TOETVA): indications, techniques and results. Surg Endosc 2018;32:456-65. [Crossref] [PubMed]
71. Bae DS. Current status of robotic thyroid surgery in South Korea: a web-based survey. World J Surg 2014;38:2632-9. [Crossref] [PubMed]
72. Shedd DP, Burget GC. Identification of the recurrent laryngeal nerve. Arch Surg 1966;92:861-4. [Crossref] [PubMed]
73. Barczyński M, Randolph GW, Cernea CR, et al. External branch of the superior laryngeal nerve monitoring during thyroid and parathyroid surgery: International Neural Monitoring Study Group standards guideline statement. Laryngoscope 2013;123:S1-14. [Crossref] [PubMed]
74. Del Rio P, Cozzani F, Rossini M, et al. Mini-invasive thyroidectomy and intraoperative neuromonitoring: a high-volume single-center experience in 215 consecutive cases. Minerva Surg 2021;76:160-4. [Crossref] [PubMed]
75. Pragacz K, Barczyński M. Evaluation of the learning curve for intraoperative neural monitoring of the recurrent laryngeal nerves in thyroid surgery. Pol Przegl Chir 2015;86:584-93. [Crossref] [PubMed]
76. Del Rio P, Nisi P, Benedicenti S, et al. Intraoperative neuromonitoring in thyroidectomy: the learning curve. Ann Ital Chir 2016;87:298-305. [PubMed]
77. Witzel K, Benhidjeb T. Monitoring of the recurrent laryngeal nerve in totally endoscopic thyroid surgery. Eur Surg Res 2009;43:72-6. [Crossref] [PubMed]
78. Duclos A, Lifante JC, Ducarroz S, et al. Influence of intraoperative neuromonitoring on surgeons' technique during thyroidectomy. World J Surg 2011;35:773-8. [Crossref] [PubMed]
79. Wojtczak B, Kaliszewski K, Sutkowski K, et al. The learning curve for intraoperative neuromonitoring of the recurrent laryngeal nerve in thyroid surgery. Langenbecks Arch Surg 2017;402:701-8. [Crossref] [PubMed]
80. Dionigi G, Bacuzzi A, Boni L, et al. What is the learning curve for intraoperative neuromonitoring in thyroid surgery? Int J Surg 2008;6:S7-12. [Crossref] [PubMed]
81. Patel A, Ally M, Venkatachalam V, et al. The learning curve and safety of continuous intraoperative vagus nerve monitoring in thyroid surgery. Ann R Coll Surg Engl 2022;104:618-23. [Crossref] [PubMed]
82. Choi HW, Ji YB, Kim E, et al. Success rate and learning curve of intraoperative neural monitoring of the external branch of the superior laryngeal nerve in thyroidectomy. Head Neck 2021;43:3946-54. [Crossref] [PubMed]