Evaluating the feasibility of a novel Marking Breast Oncoplastic Surgery Simulator (MBOSS) as a training tool for marking: a randomised trial

Introduction

Breast cancer is the top cancer affecting women worldwide. Oncoplastic surgery has allowed selected breast cancer patients with larger tumor-to-breast volume ratio to safely excise the tumor while conserving their breasts and still achieve a good cosmetic outcome (1). Re-excision rates with oncoplastic surgery were also reported to be lower compared to conventional wide local excision (2).

However, oncoplastic surgery has a steep learning curve, and skin marking for incision placement requires hands-on practice. Female volunteers are commonly recruited for the teaching and practice of oncoplastic surgery marking. However, it may be difficult to recruit these volunteers as the breast is an intimate organ (3). Also, the teaching group size must be kept small to maintain the privacy of these volunteers. To overcome these problems, simulation models such as Mastotrainer (4) has been developed. While Mastotrainer allows repeated marking, it is expensive. In addition, while Mastotrainer has the additional advantage of allowing the oncoplastic surgical steps to be performed, it is limited to a single use after the surgical steps were performed. There are other breast simulation models available but only a few are droopy in appearance or can mimic the texture of the female breast for oncoplastic surgery marking.

Consequently, we have developed a novel Marking Breast Oncoplastic Surgery Simulator (MBOSS) for the purpose of teaching oncoplastic surgery marking. We aim to evaluate its effectiveness as a training tool compared to volunteers using the Kirkpatrick evaluation model (5) with outcome measures of (I) learner satisfaction, (II) knowledge gain, (III) skills adaption and (IV) organizational impact. To the best of our knowledge, MBOSS is one of the few droopy breast models available worldwide that can mimic the appearance and texture of the female breast.

We present the following article in accordance with the CONSORT reporting checklist (available at http://dx.doi.org/10.21037/gs-20-476).

Methods

In this prospective study, breast/plastic/general surgeons and trainees, from multiple countries, were enrolled. Participants were first grouped into little/no, moderate or much oncoplastic experience, which was determined based on their pre-course self-assessment. Within each oncoplastic experience group, stratified randomization (6) was performed to assign the participants into two groups with one group using MBOSS for training and other the volunteer (Figure 1). Each teaching and assessment session consisted of 10 participants lasting about 5 hours.

Figure 1 Study design.

All participants underwent a written pre-test which required them to list in sequence the steps of the inverted T (wise pattern) mammoplasty marking. They then watched a video demonstration on how to perform the inverted T mammoplasty marking after which they underwent teaching using either MBOSS or volunteer by a single instructor. During the teaching session, each participant was given the opportunity to practice inverted T mammoplasty marking on the MBOSS or a volunteer, depending on their respective group assignment, with feedback and supervision from the instructor.

Following the training, participants in both groups were assessed for competency in the inverted T mammoplasty marking technique by a single examiner, blinded to group assignment, using a volunteer. The volunteer was used in the assessment of all participants and was not the same volunteer used in training.

All participants were required to participate in a self-report questionnaire, giving an assessment on a scale of 1–5 (5 being the best score) of their learning experience, perceived surgical knowledge acquired and competencies in performing marking compared to their pre-session status, and relevance of this training to their surgical practice.

Participants completed a written post-test assessment that was compared to their pre-test assessment to evaluate the knowledge gain. Pre- versus post-test results were evaluated by a single assessor blinded to group assignment using a reference checklist.

Participants randomized to the MBOSS training group participated in a Likert scale questionnaire assessing the physical attributes and realism of MBOSS. In addition, MBOSS participants assessed the ability of MBOSS to achieve its purpose compared to a volunteer and its value as a training tool.

Both the instructor and examiner were plastic surgeons with more than 20 years of operative experience.

The outcomes between the two groups were compared based on Kirkpatrick’s educational evaluation model. The various levels of evaluation outcomes, level 1 satisfaction, level 2 knowledge gain, level 3 skill application and level 4 organizational impact, were respectively evaluated using (I) a self-report Likert scale questionnaire, (II) pre- and post-test assessments, (III) a marking skills competency assessment and (IV) a 6-month post-course questionnaire.

Statistical analysis

Fisher’s exact test was used to compare categorical variables between the MBOSS and volunteer groups, and the 2-sample t-test or Wilcoxon rank-sum test was used as appropriate for comparing continuous variables. P<0.05 was considered statistically significant. SAS v9.4 (SAS Institute Inc., Cary, NC, USA) was used to perform all analyses.

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study obtained SingHealth Centralised institutional review approval (CIRB Ref: 2017/2841) and all participants consented to participation in the study.

MBOSS

MBOSS was designed as a vest, such that it can be easily fitted on a torso to facilitate the marking (Figure 2).

Figure 2 Side (A,C) and front (B) views of MBOSS. MBOSS, Marking Breast Oncoplastic Surgery Simulator.

MBOSS was fabricated with silicon using a mold produced by stereolithography, an efficient 3D printing technique (7). This allows MBOSS to simulate droopy breasts in appearance and texture, so that MBOSS can be maneuvered during marking in a fashion similar to that of female droopy breasts. In addition, MBOSS has a smooth surface profile which allows easy marking and marking removal, enabling repeated markings to be performed.

Results

Forty participants (77.5% females and 22.5% males) from various countries were enrolled with 57.5% between 31–40 years old. All were from the breast surgery discipline with a single participant from the plastic surgery discipline. Thirty-two-point-five percent were trainees/fellows while the rest had completed their surgical training (Table 1). There were no statistically significant differences among demographic parameters between the MBOSS and volunteer groups.

Table 1 Demographics of the 40 participants
Full table

Of those who enrolled in the course, 70% rated their oncoplastic experience (self-perceived) as “little” or “basic” and were unable to perform mammoplasty marking (Table 2). Seventy-seven-point-five percent of participants scored <50% on the pre-test. There was no significant difference in pre-test results relative to self-perceived oncoplastic experience.

Table 2 Comparison of outcomes between the participants using MBOSS versus volunteers as training tool
Full table

Using the Kirkpatrick model of evaluation, for level 1 outcomes, all participants rated favorably regarding the engagement, relevance and satisfaction with the workshop (Table 2) with no significant difference between the MBOSS and volunteer groups.

Regarding open-ended questions on areas of improvement, the more-cited suggestions included a longer workshop that included other oncoplastic marking techniques, smaller participant to model/volunteer ratio with ideally one model/volunteer to each participant, and inclusion of live surgery and more videos demonstrating the steps of the oncoplastic operation.

For level 2 knowledge gain outcomes, participants fared better in post-test results compared to pre-tests, with the MBOSS group achieving a significantly better score (P=0.0471). Participants favorably rated attitude, confidence and commitment of applying course skills to work, with no significant differences between groups (Table 2). Three participants rated their confidence level as average. The most common reasons cited for decreased confidence in applying the course skills to work were—even after completing the course—insufficient knowledge and skills, and lack of necessary resources.

When presented with open-ended questions on the anticipated barriers in applying their newly gained knowledge and skills, the commonly cited barriers included availability of suitable patients wanting breast conservation or who have ptotic breasts, insufficient support from colleagues, and insufficient knowledge on the operative steps of oncoplastic surgery.

For the level 3 skills application evaluation, one participant from the MBOSS group did not complete the assessment as she had to leave early for personal reasons. For the remaining 39 participants, marking skills assessments did not differ statistically between study groups.

Post-workshop, most participants rated themselves more favorably compared to their pre-workshop status as having greater understanding of the principles of marking and were able to perform it with assistance (32.5%), without assistance (50%) or teach others (15%). There was no significant difference in post-workshop self-ratings between study groups.

Finally, for the level 4 organizational impact evaluation, when asked what outcomes the participants expected to achieve at the beginning of the training if they applied their newly gained knowledge and skills, the top three selected outcomes was (I) increased use of oncoplastic breast surgery (90%), (II) better cosmetic outcomes (90%) and (III) increased patient satisfaction (70%). There was no significant difference in perceived initial outcomes between study groups.

A 6-month post-course level 4 evaluation revealed an overall 55% response rate, with 60% and 50% response rates for the MBOSS and volunteer groups, respectively. Of those who had responded, 58.3% and 60% of the MBOSS and volunteer groups had applied the marking skills in their practice in the last 6 months. The most commonly cited reason for no application of the marking skills was lack of opportunity.

Of the participants who had applied the marking skills, there was no significant difference in the level 4 reported outcomes. Both groups reported a very positive impact of the training on their practice with better cosmetic outcomes, increased use of oncoplastic surgery and improved patient satisfaction among the top 3 effects witnessed.

Evaluation of MBOSS

Among the 20 participants who used MBOSS in their training, on a scale of 1 (not good) to 5 (very good), 16 (80%) and 11 (55%) chose a score of 4 or 5 for MBOSS realism of appearance and texture respectively compared to the female breast. The remaining participants scored realism of appearance and texture as a 3.

Ninety percent of participants felt that the MBOSS learning experience was realistic for a score of 4 or 5 (very realistic), and 95% evaluated MBOSS as having achieved its training purpose for breast marking with a favorable score of 4 or 5.

Ninety-five percent assessed MBOSS as a good value training tool with scores of 4 or 5. Finally, 85% would consider using MBOSS instead of volunteers for teaching breast marking with a favorable score of 4 or 5.

Discussion

Our single blinded, randomized study showed that MBOSS has comparable learning satisfaction, adaptation of skills and organizational impact comparable with the use of volunteers, and higher knowledge gains, as assessed by the Kirkpatrick evaluation model for teaching inverted T mammoplasty marking. Although realism in appearance and texture of MBOSS were rated as inferior to the female breast, a majority of participants nevertheless rated MBOSS favorably as a good training tool and would consider using MBOSS instead of volunteers for teaching marking.

Oncoplastic surgery marking requires hands-on practice. Traditionally, marking has been taught primarily using female volunteers. However, as the breast is an intimate organ, it can be difficult to recruit volunteers. In addition, each marking session is usually done with a small group of participants due to privacy reasons, and the volunteer can experience participation fatigue, further limiting the number of marking each session.

To overcome the limitations associated with the use of volunteers for teaching purposes, simulation models have been used. Simulation models allow the learner to acquire the necessary skills in an environment which replicates a real-life clinical scenario with the advantages of flexible training time and unlimited attempts without harm to patients (8). Simulation models can also be used for prolonged sessions, allowing more trainees being trained. The theory of deliberate practice (9) is particularly useful in teaching the examination of intimate organs such as breast and pelvis whereby the use of simulation models has been shown to improve skills significantly compared to no intervention (3).

There are several breast simulation models but few have both a droopy appearance and a texture that mimic the female breasts. Of the few available droopy breast models with texture similar to female breast, even fewer have undergone evaluation of their usefulness. Mastotrainer is one such droopy breast simulation teaching tool previously evaluated for usefulness. It allows the comprehensive practice of oncoplastic surgery marking and surgical steps of different techniques. However, it is cost prohibitive and limited to a single practice simulation session of surgical steps. As a result, MBOSS, one of the world’s few droopy breast models, was created to provide a cheaper simulation alternative, with a cost price about half that of Mastotrainer, and yet be as effective as volunteers in teaching oncoplastic surgery marking.

An advantage of MBOSS—versus the volunteer—is that MBOSS allows a longer workshop to be planned with teaching of other oncoplastic marking techniques, without concern for volunteer fatigue. This was one of the suggestions proposed for course improvement by some of the participants taught using the volunteer. Practice may also be done at participants’ own time and pace, irrespective of the volunteer availability. In contrast to Mastotrainer for the teaching of marking only, MBOSS is not as cost prohibitive. As a result, more models can be purchased for each course, resulting in a lower participant-to-model ratio, with more dedicated individual time for repeated hands-on practice.

MBOSS is not without limitations. The texture of MBOSS cannot completely mimic that of the female breast, nevertheless it was rated favorably by more than half of participants and did not differ significantly from volunteer in skill application during the marking skill assessment. MBOSS does not allow the surgical steps of the operation to be performed. However, the surgical steps of the oncoplastic operation could potentially be taught using other simulation teaching tools, such as the virtual breast oncoplastic surgery simulator (VBOSS) (10) that uses virtual reality to allow participants to practice the steps of oncoplastic surgery.

In all surgical training courses, proper evaluation of learning outcomes is important. In this study, the Kirkpatrick evaluation model and the experimental/quasi-experimental model of the intact group design (11) with randomization were used. The Kirkpatrick evaluation model was chosen because it allows a clear focus on the course outcomes with explicit and meaningful outcome levels beyond simple learner satisfaction. However, the Kirkpatrick model does not account for learner pre-course experience. To overcome some of the shortcomings associated with Kirkpatrick model in this study, learner knowledge and skills were assessed prior to the course with matching of learners of similar experience to control for confounding factors which might affect course outcomes. An intact group study design was chosen using randomisation to MBOSS and volunteer groups to control factors among learners that might otherwise affect the course outcomes. This design has been applied previously to several medical evaluation studies, including breast examination studies (12,13).

Strengths of this paper include a formal evaluation of learning outcomes, including evaluation of marking skill application using the Kirkpatrick evaluation model—an element which is lacking in many course evaluations. Using the Kirkpatrick evaluation model, other characteristics affecting learning outcomes, such as the participant learning attitude, commitment to applying course skills etc. were evaluated as well. These characteristics should be incorporated into the evaluation survey but are often not assessed in surgical education programs. In addition, having the same instructor, examiner and volunteer for teaching and a different single volunteer for examination minimized interpersonal bias. As a single-blinded randomized study, selection and assessment bias were eliminated. There was also a 6-month post-course follow-up evaluation that demonstrated a positive translational impact of the training on patient clinical outcomes.

Limitations included a relatively small sample size. However, other breast simulation studies assessing the feasibility of a novel training tool have recruited as few as 4 patients (14). As this was a pilot study on the feasibility of novel MBOSS as a training tool, our sample size was influenced by considerations of anticipated logistical difficulties in recruiting volunteers and limitations in group sizes due to the privacy required for examination of female breasts. Another shortcoming was that study participants were initially grouped based on a subjective self-assessment of oncoplastic experience and then randomized into the teaching groups. Although a more objective assessment of the oncoplastic marking experience would have been preferred, it was not possible logistically prior to the workshop. However, the self-assessment was reflective overall of participant experience as shown by the pre-test results. Only a single oncoplastic marking technique was assessed using MBOSS in this study, though inverted T mammoplasty marking was the commonly used technique (15,16) in some centers. MBOSS could also be used to teach other marking techniques, such as vertical mammoplasty and round block mastopexy etc., but these were not assessed during this course due to time constraints. The response rate at the 6-month post-course follow-up was also suboptimal.

In conclusion, MBOSS is a novel and effective teaching model that can be used as an alternative to volunteers in teaching mammoplasty marking. MBOSS offers a cheaper alternative to existing droopy breast models, overcomes logistical issues associated with volunteer recruitment and allows training to be done repeatedly at one’s own pace.

Acknowledgments

The preliminary results of this study were accepted for poster presentation at the American Society of Breast Surgeons 21st Annual Meeting 2020.

Funding: The authors would like to acknowledge the financial support from the National Additive Manufacturing Innovation Cluster (NAMIC) grant (project ID: 2017275). This evaluation study was funded by the Academic Medicine Education Institute (AM•EI) Education Grant 2018.

Footnote

Reporting Checklist: The authors have completed the CONSORT reporting checklist. Available at http://dx.doi.org/10.21037/gs-20-476

Data Sharing Statement: Available at http://dx.doi.org/10.21037/gs-20-476

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/gs-20-476). GHL received Academic Medicine Education Institute (AM.EI) education grant 2018 for the conduct of the study. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study obtained SingHealth Centralised institutional review approval (CIRB Ref: 2017/2841) and all participants consented to participation in the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Lim G, Pineda LA. Applicability of Oncoplastic Breast Conserving Surgery in Asian Breast Cancer Patients. Asian Pac J Cancer Prev 2016;17:3325-8. [PubMed]
2. Losken A, Dugal CS, Styblo TM, et al. A meta-analysis comparing breast conservation therapy alone to the oncoplastic technique. Ann Plast Surg 2014;72:145-9. [PubMed]
3. Dilaveri CA, Szostek JH, Wang AT, et al. Simulation training for breast and pelvic physical examination: a systematic review and meta-analysis. BJOG 2013;120:1171-82. [PubMed]
4. Zucca-Matthes G, Lebovic G, Lyra M. Mastotrainer new version: realistic simulator for training in breast surgery. Breast 2017;31:82-4. [PubMed]
5. Kirkpatrick D. Revisiting Kirkpatrick’s four level model. Train Dev 1996;1:54-9.
6. Suresh K. An overview of randomization techniques: An unbiased assessment of outcome in clinical research. J Hum Reprod Sci 2011;4:8-11. [Crossref] [PubMed]
7. Tappa K, Jammalamadaka U. Novel Biomaterials Used in Medical 3D Printing Techniques. J Funct Biomater 2018;9:17. [Crossref] [PubMed]
8. Evgeniou E, Loizou P. Simulation-based surgical education. ANZ J Surg 2013;83:619-23. [Crossref] [PubMed]
9. Ericsson KA, Krampe RT, Tesch-Roemer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev 1993;100:363-406. [Crossref]
10. Lim GH, Lee J, Yen CC. Virtual breast oncoplastic surgery simulator (VBOSS): A novel training tool in breast surgery. Eur J Surg Oncol 2018;44:1750-3. [Crossref] [PubMed]
11. Frye AW, Hemmer PA. Program evaluation models and related theories: AMEE guide no. 67. Med Teach 2012;34:e288-99. [Crossref] [PubMed]
12. Campbell HS, Fletcher SW, Pilgrim CA, et al. Improving physicians' and nurses' clinical breast examination: a randomized controlled trial. Am J Prev Med 1991;7:1-8. [Crossref] [PubMed]
13. Pilgrim C, Lannon C, Harris RP, et al. Improving clinical breast examination training in a medical school: a randomized controlled trial. J Gen Intern Med 1993;8:685-8. [PubMed]
14. Kazan R, Courteau B, Cyr S, et al. A Novel Mammoplasty Part-Task Trainer for Simulation of Breast Augmentation: Description and Evaluation. Simul Healthc 2016;11:60-4. [PubMed]
15. McCulley SJ, Macmillan RD. Therapeutic mammaplasty--analysis of 50 consecutive cases. Br J Plast Surg 2005;58:902-7. [PubMed]
16. Fitoussi AD, Berry MG, Famà F, et al. Oncoplastic breast surgery for cancer: analysis of 540 consecutive cases Plast Reconstr Surg 2010;125:454-62. [outcomes article]. [Crossref] [PubMed]