Thyroid surgery YouTube videos: estimating quality by surgeon characteristics and view rate

Introduction

It is not known how often medical students and residents turn to YouTube video content to learn about and review medical information or techniques. It is known that medical education in the United States is becoming increasingly digital. In many cases online content that is available anytime and anywhere on laptops, tablets, and smart phones is replacing the use of traditional textbooks (1). The integration of multimedia and information is natural for many current residents and medical students as they are among the first generation of physicians who have had access to the internet throughout their higher education.

One of the primary issues in utilizing publically available online materials as a source of medical education is the lack of transparent peer review. While many journals and textbooks that provide online editions and supplements maintain rigorous standards, publicly available websites and information can be written or posted by anyone, without editing or fact checking. As an example, numerous videos demonstrating surgical procedures are freely available on YouTube. The originators are not required to provide credentials, may remain completely anonymous, are not compelled to cite sources, and the content is not edited or peer-reviewed prior to posting.

Several recent studies have analyzed the educational value and accuracy of videos found on YouTube addressing topics such as physical exam, disease-specific information, and medical treatments (2-4). Specific to the field of otolaryngology, two studies have evaluated YouTube videos as a source of general information for tonsillectomy and vent tube placement. Both studies found that less than 25% of videos reviewed contained information that was both accurate and helpful (5,6).

There are studies published in surgical fields that show a positive effect on performance of a procedure after watching an instructional YouTube video as well as studies that analyze the quality of procedural instruction presented (2,7,8). This suggests that the value of such materials has great potential to improve surgical education and patient outcomes. Many of these papers differentiate between physician-sourced content and content posted by industry or patients (5,6,9). In this relatively new field of study there has not been a deeper analysis of the physicians who are uploading videos to YouTube.

In this study we performed a search for a surgical procedure (thyroidectomy) on YouTube, similar to the search a trainee might perform to supplement and corroborate more traditional educational content within his/her training environment. Our aims were to: (I) identify the source of posted surgical videos of thyroidectomy procedures on YouTube; (II) of the surgeon-sourced content, categorize the surgeons’ national origin, specialty, practice setting, history of publication in the field, h-index, and the type of surgery performed; and (III) compare surgeon characteristics with the number of views for each video.

Methods

We performed a search on the standard YouTube search engine for the term “thyroidectomy” with the filter set to sort videos by relevance (the default YouTube setting). Data were gathered for the first one hundred videos available. One hundred was chosen to reflect how far a typical surgical trainee would be likely to delve into the YouTube “hits” in the interest of time and convenience. Data recorded included video type (live surgical, informational, patient testimonial or other), the name of the primary surgeon, the number of views, and the type of surgery performed (open vs. endoscopic/robotic). Videos were considered “live surgical” if the majority of the video contained footage of thyroid surgery being performed on an actual patient.

Each surgeon who posted a live surgical video and/or was the operating surgeon in the video, was searched on Scopus to ascertain h-index and thyroid-related publication history. A general internet search was performed for each surgeon to determine the location of their practice and whether they maintain affiliation with an institution that trains surgical residents or fellows.

Data were summarized using proportions, means, ranges and 95% confidence intervals. Chi-squared tests were performed to compare proportions of the categorical variables. h-index data were categorized as 0–3 and >3. The number of views was analyzed with one way ANVOVA and follow-up t-tests. Statistical analysis was performed with Microsoft Excel.

Results

There were 7,260 video results for the search term “thyroidectomy.” The first 100 videos were analyzed. Of the top 100 videos, 62 were live surgical videos, 26 were patient testimonials, 10 were informational, and 2 were veterinary surgical videos. A primary surgeon was identified in 58 of the 62 (94%) live surgical videos. Twenty-eight of the 58 (48%) surgical procedures were robotic and/or endoscopic, and the remaining procedures were traditional access/open thyroidectomy.

The 58 videos with an identifiable primary surgeon originated from 21 different countries, most often from the United States and India as summarized in Table 1. Forty-two unique primary surgeons were identified with 6 videos being the most posted by or about any individual surgeon. Of the 42 surgeons, 13 were otolaryngologists and 29 were general surgeons. Otolaryngologists were primary surgeon in 16 of the 58 (28%) surgical videos and general surgeons in 42 of 58 (72%).

Table 1 Origin of videos by country
Full table

Of the 42 surgeons, 18 (43%) had at least one publication related to the thyroid. Twenty of the 42 (48%) surgeons had an h-index greater than 3 with an overall average of 10 and a range from 0 to 56. Academic affiliation was identified for 32 of the 42 (76%) surgeons and academically affiliated surgeons accounted for 43 of 58 (74%) videos.

Surgeon characteristics were analyzed and statistically significant findings are summarized in Tables 2-5. Of note, US surgeons featured in thyroidectomy videos were more likely to have an h-index >3, have published a thyroid-related paper, be an otolaryngologist, or perform endoscopic or robotic-assisted thyroid surgery. Those posting or featured in a video of minimal-access thyroid surgery were more likely to have published a thyroid-related paper or have an academic affiliation. There were no differences between otolaryngologists and general surgeons in their proportion of academic affiliation, their rate of thyroid-related publication, the proportion of endoscopic/robotic procedures, or the rate of h-index >3. There was also no difference in the rate of academic affiliation between surgeons from the US and the rest of the world.

Table 2 Significant differences in surgeon nationality
Full table

Table 3 Significant differences in surgeon h-index
Full table

Table 4 Significant differences in endoscopic surgeons
Full table

Table 5 Significant differences in surgeon thyroid-related publications
Full table

The average number of views for live surgical videos was significantly higher than patient testimonials, 16,204 to 2,011 (P<0.05). However, there was no statistical difference between either group and the informational group which had an average of 8,742. The most-watched video had 283,631 views. There were no statistically significant correlations between the number of views and any of the surgeon characteristics analyzed, summarized in Table 6.

Table 6 Comparison of average number of views by surgeon characteristics
Full table

The four most-watched videos had more views than the other 96 videos combined. They were all videos of open surgical procedures and had 716,230 cumulative views compared to 431,356 total views for the remaining videos. One of the four videos was in the first five search results and three were in the top twenty. The most viewed video was a non-academic otolaryngologist in the US, the second was an academic general surgeon operating in Djibouti, the third was a non-academic general surgeon in the Philippines, and the fourth was an academic otolaryngologist in the US.

Discussion

Publically available live surgical videos posted online have proliferated in recent years. YouTube has emerged as the prototype for video-based information about virtually anything, including live surgery. The motivation for posting a live surgical video on YouTube is unclear and may vary: is the video intended to educate other practicing surgeons, residents or patients? Is it meant to serve as a marketing tool? In addition to the general public and patients, surgical trainees likely comprise a significant portion of the audience for this material. Whereas there are many advantages to open content, including immediate access, portability, no cost, abundance and variety, the downside is the lack of quality control. There is no peer-review for YouTube surgical videos and thus no traditional mechanism to control quality or confirm that the content reflects evidence-based medicine standards of practice.

In the absence of peer review, there may be other surrogates for quality of YouTube surgical videos. Two potential quality surrogates were explored in this study: characteristics of the source (surgeons who posted or were featured in the videos), and response of the audience (view or “hit” rate of the videos).

Most of the thyroid surgery videos came from surgeons affiliated with academic institutions, suggesting that the motivation for posting the video is to educate. Whether or not the surgeon-source is an expert in thyroid surgery, thus elevating the educational value of the content, is difficult to directly measure. Available measures of academic productivity such as h-index, and thyroid-specific academic activity such as publications may suggest intellectual expertise, but not necessarily technical expertise. In contrast, superb technical expertise may be offered in a video posted by someone without academic affiliation or productivity.

Response of the audience as measured by view rate is another surrogate for quality. Limitations to this measurement are lack of information about the audience; how many of the viewers are surgical residents seeking expert content? Similarly, it is unknown how the views are being utilized: to glean primary knowledge by the novice, or to quickly review before a case by the more experienced surgeon. The advantage to using view rate as a quality measure is its immediate availability to the audience, and its numerical nature. However, view rate may only reflect popularity or a link from an outside website or advertisement and not the quality of the video.

This study reveals that there is an abundance of live thyroid surgical videos on YouTube. It is safe to assume that surgical trainees comprise at least some of the viewing audience, and impact on surgical education may be significant but unmeasured at present. Value of the content can be estimated by characteristics of the surgeon-sources, and by audience response. However, both of these measures are imperfect. Research in this area is in its nascence, and future investigations are warranted. Measuring the impact of YouTube surgical videos on surgical education by polling or studying the learners will be of great value, especially to determine whether this content fills an unmet need in surgical education. Additionally, developing a rating system from both the learner’s and the expert educator’s viewpoints can facilitate critical analysis of content, while avoiding the limitations of imposing peer-review mechanisms.

Conclusions

Videos of live thyroid surgery publically available on YouTube most often originated in the US, with India as a close second in global origin. General surgeons were more likely to post or be featured in thyroid surgery videos than otolaryngologists overall, but videos from the US were more likely to be from otolaryngologists, and videos from otolaryngologists trended toward more views. Most of the videos (74%) featured academically affiliated surgeons.

Videos from the US were more likely to originate from or feature a surgeon with an h-index >3, a thyroid-related publication, who is an otolaryngologist, and who performs minimal-access thyroid surgery in the video, as opposed to videos from non-US surgeons. Those surgeons performing minimal-access surgery were more likely to have published a thyroid-related paper and be affiliated with an academic institution.

YouTube video view rates were concentrated toward four videos, the view rate of which exceeded the remaining studied videos combined. There was a trend toward higher view rates of otolaryngologist-sourced videos, videos from surgeons with h-index >3, and those from US surgeons, but trends toward lower view rates from surgeons with thyroid-related publications and academic affiliation. However, there was no statistically significant predictor of higher view rate among the surgeon characteristics.

Acknowledgments

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Gutmann J, Kühbeck F, Berberat PO, et al. Use of learning media by undergraduate medical students in pharmacology: a prospective cohort study. PLoS One 2015;10:e0122624. [Crossref] [PubMed]
2. Sampson M, Cumber J, Li C, et al. A systematic review of methods for studying consumer health YouTube videos, with implications for systematic reviews. PeerJ 2013;1:e147. [Crossref] [PubMed]
3. Kerber KA, Burke JF, Skolarus LE, et al. A prescription for the Epley maneuver: ? Neurology 2012;79:376-80.www.youtube.com
4. Camm CF, Sunderland N, Camm AJ. A quality assessment of cardiac auscultation material on YouTube. Clin Cardiol 2013;36:77-81. [Crossref] [PubMed]
5. Strychowsky JE, Nayan S, Farrokhyar F, et al. YouTube: a good source of information on pediatric tonsillectomy? Int J Pediatr Otorhinolaryngol 2013;77:972-5. [Crossref] [PubMed]
6. Sorensen JA, Pusz MD, Brietzke SE. YouTube as an information source for pediatric adenotonsillectomy and ear tube surgery. Int J Pediatr Otorhinolaryngol 2014;78:65-70. [Crossref] [PubMed]
7. Rowse PG, Ruparel RK, AlJamal YN, et al. Catering to millennial learners: assessing and improving fine-needle aspiration performance. J Surg Educ 2014;71:e53-8. [Crossref] [PubMed]
8. Fischer J, Geurts J, Valderrabano V, et al. Educational quality of YouTube videos on knee arthrocentesis. J Clin Rheumatol 2013;19:373-6. [Crossref] [PubMed]
9. Bezner SK, Hodgman EI, Diesen DL, et al. Pediatric surgery on YouTube™: is the truth out there? J Pediatr Surg 2014;49:586-9. [Crossref] [PubMed]