Evaluation of YouTube videos as a patient education source for novel surgical techniques in thyroid surgery

Introduction

YouTube has become the largest open-access, video-sharing platform available, gaining over 1.9 billion active monthly users worldwide since its inception in 2005 (1). The interface allows virtually unlimited, free video content for users to view, post comments on, and to indicate likes and dislikes. Little censorship exists from the parent company, and ability to view certain content is only restricted by age verification, regional setting, and parental guidance presets. Medical and health-related content is readily found on YouTube. However, due to a lack of peer review or a verification process for medical information, YouTube videos are at risk of delivering outdated or inaccurate information regarding certain diseases and surgical procedures. Despite its global audience and popular usage, YouTube videos may not contain adequate physician or patient education materials for new surgical techniques (2,3).

A conventional cervical thyroidectomy is performed for indications of thyroid cancer and several benign thyroid diseases, including multinodular goiter, Graves’ disease, and Hashimoto’s thyroiditis. The thyroidectomy commences with the surgeon first making a low-neck incision. But minimally invasive surgical techniques to avoid a visible neck incision have been evolving, with patients and surgeons having a growing interest in these procedures (4). Robotic thyroid surgery has been done using several different approaches: transaxillary, transoral, bilateral/unilateral axillo-breast, and retroauricular. Transoral robotic thyroid surgery (TORT or TORS) is one of the latest techniques for removing the thyroid and parathyroid glands with robot-assistance. The transoral endoscopic thyroid and parathyroid vestibular approach (TOETVA or TOEPVA) surgery uses a similar incision location to TORT; however, it uses laparoscopic instruments for organ removal. Compared to TOETVA, TORT provides the advantage of a three-dimensional, magnified view; use of articulating instruments; and the ability to control an additional port by the surgeon.

The latest procedural techniques such as the ones described above may be attractive to patients who wish to avoid having a visible scar in the neck after thyroid surgery. Some of the most readily accessible and free sources of medical information for patients to explore are available through the internet. However, insufficient and less comprehensive internet-based information on novel techniques, such as TORT and TOETVA surgeries, may dissuade patients from undergoing the procedure or set misguided expectations for the procedure.

We evaluated the popularity, quality, and reliability of robotic and endoscopic transoral thyroid surgery videos on YouTube using the video power index (VPI), Global Quality Scoring (GQS), and the DISCERN Instrument scoring system. We assessed video content for the presence of procedure-specific information such as risks, benefits, operative details, and peri-operative instructions.

Methods

On March 26, 2019, search results from two keyword phrases, “robotic thyroid surgery” and “transoral thyroid surgery”, were compiled using a hypertext preprocessor (PHP) code with a YouTube application program interface (API) developer key. After running the code in Visual Studio Code software with a retrieved YouTube data API developer key, an output of a user interface consisting of modes of selections for input, sorting, and a number of results were produced. Once the inputs were submitted to the user interface window generated by the code, a list of results was generated by the API calling into the YouTube search engine (Figure 1). The queries included the top 50 video results sorted by relevance. The videos mostly consisted of robotic and endoscopic transoral thyroid surgery, robotic transaxillary thyroid surgery, and two videos on robotic retroauricular thyroid surgery. No video using bilateral axillo-breast approach for thyroid surgery was assessed. Cookies were deleted prior to the search queries. The source, duration, days since upload, view count, number of comments, and like and dislike counts were tabulated. If comments were blocked, no value was recorded.

Figure 1 PHP code output. PHP, hypertext preprocessor (HTML-embedded scripting language).

The videos were broadly categorized by upload sources such as medical professional, academic institution, commercial, and patient testimonials. Two independent medical professionals, a post-graduate medical student and a graduate researcher, scored each video using the GQS and DISCERN scoring system criteria. Additional parameters determined by the reviewers were viewer intent of the video, putative target audience, demonstration of surgical technique, and involvement of an actual patient. Viewer intent of the video was appraised by the reviewers into categories such as choice of surgical approach, preoperative preparation, postoperative, or educational material for health-care professionals. Choice of surgical approach was marked when the video contained a discussion of different surgical options. Preoperative preparation was defined as the video addressing what to expect before the scheduled surgery. Postoperative was selected when the video discussed complications after surgery. Educational purpose was indicated when the video discussed medical terminology as second nature and was addressed to healthcare professionals. Putative target audience was categorized into three groups: patients, healthcare professionals, or both. Presence in the video of an array of clinical content including pre-operative preparation, procedure demonstration, surgical technique discussion during the demonstration, postoperative care, or postoperative complications was verified by the reviewers. Videos with a significant discrepancy in scores were reassessed by a second-viewing of the videos to reach a consensus score.

Popularity analysis

The popularity of videos was evaluated with a VPI, which was first described in a YouTube video study on kyphosis (5). Popularity was assessed by VPI (like ratio × view ratio/100), where like ratio represents the like/(like + dislike) percentage, while the number of views per day determined the view ratio. A higher score indicated more popularity for the video.

Quality and reliability assessment

The GQS system, a 5-point scale rating, was first introduced by Bernard et al. to evaluate the educational quality of videos (Table 1) (6). The DISCERN instrument was applied to assess the quality of the medical information and surgical options provided in each video (Table 2) (7). The DISCERN questionnaire includes 16 individual items, each item on a 5-point scale. We modified the DISCERN questionnaire to 15 quality and reliability questions, with a final score ranging from 15 to 75.

Table 1 Global Quality Scoring (GQS) system
Full table

Table 2 DISCERN instrument scoring system
Full table

Statistical analysis

GraphPad Prism 8 software and SPSS version 23.0 were used for statistical analysis. Quantitative study variables were tested for normality using the Shapiro-Wilk test. Central tendency and dispersion parameters of the data in the form of standard deviation and interquartile ranges were estimated. Mann-Whitney U and Kruskal-Wallis tests were applied for non-parametric variables. Binary logistic regression analysis was performed, and odds ratio (OR) and 95% confidence interval (95% CI) were estimated. Statistical significance was set at P<0.05.

Results

Descriptive characteristics related to the top 50 videos from the keyword queries are shown in Figure 2 and Table 3. Assessment of the videos revealed that most videos were uploaded by medical professionals (75.8%). The majority of uploaded videos were from the United States (US) (63.6%), as search queries in English were used and Internet Protocol (IP) address was based in the US (Figure 2A,B). The 50 videos pertaining to robotic thyroid surgeries and transoral thyroid surgeries added up to a total run time of 8.45 hours with a mean ± standard deviation (SD) duration of 8.1±3.7 minutes per video (Table 3). The videos had been viewed 261,440 times to date and the mean ± SD interval since upload was 3.6±2.6 years. All videos were in English. The videos collectively had received a total of 80 comments, 1,045 likes and 203 dislikes. Procedure information was available in 89.4% of the videos, details of the surgical techniques were discussed in 63.6%, and actual footage of surgery was demonstrated in 53% of the videos (Figure 2E). Postoperative complications were mentioned in 31.8% of the videos, postoperative care instructions in 24.3%, and pre-operative preparation instructions in 12.1%. Perceived intentions of the videos were 15.2% for educational purposes, 10.6% for patients who already underwent surgery, 3% for patients who are scheduled for surgery, and 3% for potential patients making a decision on surgery (Figure 2C). Videos that did not specify one category as perceived intent or fit multiple categories accounted for 68.2%. Putative target audience was 47% for healthcare professionals, 42.4% for patients, and 10.6% for both (Figure 2D).

Figure 2 Descriptive statistics on the YouTube videos. (A) Uploading source distributes amongst academic, medical, commercial, and patient. (B) Region of publisher is the video’s origin. (C) Viewer intent describes viewer’s position. (D) Putative target audience of video’s viewers. (E) Clinical content covers key components discussed in the videos.

Table 3 Effectiveness measures for YouTube videos
Full table

Quality assessment median values and interquartile ranges were 1.9 (0.5–3.7) for VPI, 3.0 (2.0–4.0) for GQS, and 2.8 (2.3–3.2) for DISCERN score (Table 3). The benefits score from the DISCERN scoring had a median and interquartile range of 3.0 (1.0–4.6), while the risk score was 1.0 (1.0–2.0) (Table 2). GQS and DISCERN scores were positively correlated (r=0.876, P≤0.001) (Table 4).

Table 4 Correlation analysis between videos characteristics and assessment scores
Full table

No significant difference in popularity was found among the videos. However, higher quality and reliability-scoring videos were uploaded by academic institutions and contained less medical jargon to explain the procedure, perioperative care, and postoperative complications (Table 5). The linear regression model suggested that the videos with accurate procedural details or postoperative complication information significantly increased the educational quality and reliability, as they were most frequently found to be absent in the videos (Table 6).

Table 5 Association of video characteristics with popularity, quality, and reliability scores
Full table

Table 6 Independent predictor for higher quality and reliability of YouTube videos
Full table

Discussion

YouTube, as one of the most widely-used video sharing social platforms, is growing in daily uploads and popularity in the medical community (8). The number of patients who are utilizing online searches to enhance their understanding and aid decision-making for health-related purposes is increasing as well (9). Unfortunately, patients who rely on YouTube videos as their primary source of medical information may be influenced by biased or outdated information.

The first YouTube video study for assessing the quality and reliability of medical information was conducted by Keelan et al. (10). Several other studies that followed also showed consistently lower reliability and quality of medical videos (5,11-14). Videos produced by academic institutions discussing specific procedures, perioperative care instructions, risks/benefits, and those focusing on actual patient scenarios were deemed the highest in quality and reliability.

Robot-assisted thyroid surgeries continue to evolve and gain acceptance because of the equivalent complication rates and clear cosmetic superiority to traditional cervical approaches (15-20). However, robotic thyroid surgeries are still not widely performed. A majority of studies on robotic thyroid surgery are from South Korea and are based on patients who have lower body mass index and who may be undergoing thyroid surgeries for different indications from those patients with the most frequent indications for thyroidectomy in the US (21-24). In the US, robot-assisted thyroidectomy had seen a steady increase in volume from 2009 to early 2011, but then declined from mid-2011 to 2013 (25). The downtrend in the number of cases has been attributed to higher costs for robotic surgery, the steep learning curve to master the advanced technique, and the overall higher complication rate at lower-volume centers (25,26). A few meta-analyses since have shown robot-assisted transaxillary thyroidectomy to be as effective and safe as conventional cervical and endoscopic approaches when performed by high-volume surgeons (27,28).

A majority of the videos with a high VPI demonstrated lower overall quality and reliability scores. Patient-targeted videos with higher scores included sufficient procedure details as well as preoperative and postoperative instructions. Markers of high-quality videos for patient understanding included the following: a professional introduction with relevant academic and hospital credentials; the addition of visual aids for procedural content; live technique demonstrations; inclusion of medical terminology definitions; presenters speaking in concise, spaced intervals; and discussion of frequently asked questions including risks/complications, postoperative care instructions, and citation of references and resources.

Limitations of our study include a small number of reviewed videos for each keyword and inability to account for geographical bias resulting from the IP address and language selection. Transoral endoscopic thyroid surgeries have been performed over the longest period of time in Thailand, and TORT is rarely performed outside of South Korea. Procedural and perioperative care instruction videos intended for Thai and Korean patients may be of higher informational quality than the videos that were found in English. Another shortcoming of this study is that it used only two video reviewers who both have a medical background; thus, judging was not representative of the general public. The introduction of subjective bias in the video scoring may be minimized in future studies by the inclusion of more reviewers of varying backgrounds such as healthcare consumers, medical students, and representative age groups.

Conclusions

The medical information presented in YouTube videos pertaining to robotic thyroid surgery and transoral thyroid surgery scored low in quality and reliability as patient education material. Physicians should aim to increase the quality and reliability of the videos by providing better explanations of risks for patient understanding, using visual aids, and discussing postoperative complications and care instructions to support patients’ decisions on novel surgical techniques for thyroid surgery.

Acknowledgments

The study was graciously accepted as a poster presentation at the 89th Annual Meeting of the American Thyroid Association. The authors thank Karla D. Passalacqua, PhD, for editorial assistance.

Funding: None.

Footnote

Data Sharing Statement: Available at http://dx.doi.org/10.21037/gs-20-734

Peer Review File: Available at http://dx.doi.org/10.21037/gs-20-734

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/gs-20-734). EK serves as an Editor-in-Chief of Gland Surgery. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. YouTube by the numbers: stats, demographics, etc. Available online: (Accessed on July 12, 2019).https://www.omnicoreagency.com/youtube-statistics/
2. Pandey A, Patni N, Singh M, et al. YouTube as a source of information on the H1N1 influenza pandemic. Am J Prev Med 2010;38:e1-3. [Crossref] [PubMed]
3. Sood A, Sarangi S, Pandey A, et al. YouTube as a source of information on kidney stone disease. Urology 2011;77:558-62. [Crossref] [PubMed]
4. Koh DH, Jang WS, Park JW, et al. Efficacy and safety of robotic procedures performed using the da Vinci Robotic Surgical System at a single institute in Korea: experience with 10000 cases. Yonsei Med J 2018;59:975-81. [Crossref] [PubMed]
5. Erdem MN, Karaca S. Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube. Spine (Phila Pa 1976) 2018;43:E1334-9. [Crossref] [PubMed]
6. Bernard A, Langille M, Hughes S, et al. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007;102:2070-7. [Crossref] [PubMed]
7. DISCERN: quality criteria for consumer health information. Available online: (Accessed March 30, 2019).http://www.discern.org.uk/
8. Desai T, Shariff A, Dhingra V, et al. Is content really king? An objective analysis of the public’s response to medical videos on YouTube. PLoS One 2013;8:e82469. [Crossref] [PubMed]
9. Tan SS, Goonawardene N. Internet health information seeking and the patient-physician relationship: a systematic review. J Med Internet Res 2017;19:e9. [Crossref] [PubMed]
10. Keelan J, Pavri-Garcia V, Tomlinson G, et al. YouTube as a source of information on immunization: a content analysis. JAMA 2007;298:2482-4. [Crossref] [PubMed]
11. Madathil KC, Rivera-Rodriguez AJ, Greenstein JS, et al. Healthcare information on YouTube: a systematic review. Health Informatics J 2015;21:173-94. [Crossref] [PubMed]
12. Ferhatoglu MF, Kartal A, Ekici U, et al. Evaluation of the reliability, utility, and quality of the information in sleeve gastrectomy videos shared on open access video sharing platform YouTube. Obes Surg 2019;29:1477-84. [Crossref] [PubMed]
13. Oremule B, Patel A, Orekoya O, et al. Quality and reliability of YouTube videos as a source of patient information on rhinoplasty. JAMA Otolaryngol Head Neck Surg 2019;145:282-3. [Crossref] [PubMed]
14. Abu Daabes AS, Kharbat FF. A content analysis of Arabic YouTube videos for cancer treatment. International Journal of Health Governance 2019;24:267-73. [Crossref]
15. Hüscher CS, Chiodini S, Napolitano C, et al. Endoscopic right thyroid lobectomy. Surg Endosc 1997;11:877. [Crossref] [PubMed]
16. Chang EHE, Kim HY, Koh YW, et al. Overview of robotic thyroidectomy. Gland Surg 2017;6:218-28. [Crossref] [PubMed]
17. Lang BH, Wong CK, Tsang JS, et al. A systematic review and meta-analysis comparing surgically-related complications between robotic-assisted thyroidectomy and conventional open thyroidectomy. Ann Surg Oncol 2014;21:850-61. [Crossref] [PubMed]
18. Adam MA, Speicher P, Pura J, et al. Robotic thyroidectomy for cancer in the US: patterns of use and short-term outcomes. Ann Surg Oncol 2014;21:3859-64. [Crossref] [PubMed]
19. Lee S, Lee CR, Lee SC, et al. Surgical completeness of robotic thyroidectomy: a prospective comparison with conventional open thyroidectomy in papillary thyroid carcinoma patients. Surg Endosc 2014;28:1068-75. [Crossref] [PubMed]
20. Tae K, Song CM, Ji YB, et al. Comparison of surgical completeness between robotic total thyroidectomy versus open thyroidectomy. Laryngoscope 2014;124:1042-7. [Crossref] [PubMed]
21. Axente DD, Constantea NA. Robot-assisted transaxillary thyroid surgery--retrospective analysis of anthropometric features. Langenbecks Arch Surg 2016;401:975-81. [Crossref] [PubMed]
22. Kang SW, Lee SC, Lee SH, et al. Robotic thyroid surgery using a gasless, transaxillary approach and the da Vinci S system: the operative outcomes of 338 consecutive patients. Surgery 2009;146:1048-55. [Crossref] [PubMed]
23. Dionigi G. Robotic thyroidectomy: Seoul is not Varese. Otolaryngol Head Neck Surg 2013;148:178. [Crossref] [PubMed]
24. Lin HS, Folbe AJ, Carron MA, et al. Single-incision transaxillary robotic thyroidectomy: challenges and limitations in a North American population. Otolaryngol Head Neck Surg 2012;147:1041-6. [Crossref] [PubMed]
25. Noureldine SI, Jackson NR, Rufano RP, et al. A comparative North American experience of robotic thyroidectomy in a thyroid cancer population. Langenbecks Arch Surg 2013;398:1069-74. [Crossref] [PubMed]
26. Hinson AM, Kandil E, O'Brien S, et al. Trends in robotic thyroid surgery in the United States from 2009 through 2013. Thyroid 2015;25:919-26. [Crossref] [PubMed]
27. Terris DJ, Singer MC. Qualitative and quantitative differences between 2 robotic thyroidectomy techniques. Otolaryngol Head Neck Surg 2012;147:20-5. [Crossref] [PubMed]
28. Jackson NR, Yao L, Tufano RP, et al. Safety of robotic thyroidectomy approaches: meta-analysis and systematic review. Head Neck 2014;36:137-43. [Crossref] [PubMed]