Background

Over the past several decades, simulation has become more widely used as a teaching modality in hospitals, dedicated simulation centers, mobile simulation centers, and in situ environments [1]. There has been a particular increase in simulation as a mode of instruction in medical student and postgraduate medical education [2]. Simulation-based training (SBT) is defined as the artificial representation of a real-world process in an attempt to achieve educational goals through experiential learning [3]. SBT within the medical sphere covers a variety of skills and functions, and often involves the use of task trainers or synthetic models of specific anatomic structures to teach a wide range of procedures [4,5,6]. Studies have shown that learners who participate in simulation will gain increased competence and confidence with additional practice and experience [7, 8].

In addition, all levels of learners in medical education seem to benefit from SBT. Current literature shows that residents and medical students can learn a wide range of skills, including transvaginal ultrasonography, vascular access, cardiac life support, thoracentesis, and basic laparoscopic surgical techniques [9,10,11,12,13].

As technology has advanced, more attention has focused on introducing virtual reality (VR) into simulation in medical education programs around the world. VR simulation uses computers and human simulations to create realistic and immersive learning that can be applied to a variety of medical and surgical specialties [7, 14,15,16]. The utilization of this technology inspires a new paradigm for teaching students how to maximize success in the clinical arena.

Near-peer learning was originally described as the use of more academically advanced peers teaching fellow students in higher education settings [17], and has demonstrated utility in a variety of medical education areas including objective structured clinical skills (OSCE) preparation, ultrasound image interpretation, and cardiopulmonary resuscitation [18,19,20]. Potential benefits noted in this type of training include improved learning in both students and near-peer teachers, perceived safer learning environments, developing skills for future educators, providing role models, and alleviating faculty teaching burden [21, 22].

While there are studies illustrating the validity and versatility of simulation [23, 24] and VR training for performing procedures, there is limited data available assessing their usage as standardized tools in medical education. The primary aim of this study was to assess the feasibility and utility of two innovative training modalities, anatomic VR simulation and near-peer student-led simulation training, to teach medical students technical skills such as the lumbar puncture (LP) procedure. The secondary aim was to determine students’ perceptions of the VR versus student-led simulation training modalities to elucidate how their knowledge and confidence in performing lumbar punctures in future clinical settings changed following their training.

Methods

This study was conducted at a nationally accredited medical college that has over 320 students and over 1600 total faculty members that support its mission. The project was approved by the medical college’s institutional review board (IRB) prior to initiation.

Student-Led Simulation Program

The mainstay of clinical skills development at our institution, outside of select skills labs built into curricular time, is the Student Led Independent Procedure Simulations (SLIPS) program which is a type of near-peer training. In this program, student leaders teach their peers how to perform various clinical procedure skills such as suturing, lumbar puncture, central vascular access, paracentesis, and thoracentesis. Student leaders are trained in how to perform and teach the procedures from both faculty and online educational trainings. Participants of the SLIPS program watch independent learning modules and videos of the targeted procedure prior to participating in direct hands-on instruction with the SLIPS leaders. The simulation curriculum at our institution also integrates high-fidelity and low-fidelity simulation–based training, in addition to VR and augmented reality (AR) modules, in its baseline undergraduate medical curriculum.

VR System and Equipment

The Simulation Curriculum Director of the College of Medicine designed a VR lumbar puncture training module from a lumbosacral MRI using Arivis InViewR patent pending technology (AG Imaging Science Unit, Phoenix, AZ). Consent was obtained from the subject for the full use of their MRI in this project.

This training module allowed students to visualize the spine in multiple dimensions and remove layers of skin, subcutaneous tissue, and ligaments via manual controllers to learn anatomic and spatial relationships. Haptic feedback was elicited if students punctured the ligamentum flavum into the lumbar cistern, touched vertebrae, or progressed too deep into the spinal column. For the task trainer portion of the study, commercially available standard adult LP task trainers were utilized (Simulab; Washington).

Participant Recruitment

Medical students currently enrolled at the University of Arizona College of Medicine - Phoenix (UACOM-P) were solicited for participation via the school’s email listserv. The invitation for participation was directed toward students who were unlikely to have had any prior experience with performing lumbar punctures on any level. The participant group included first-year medical students and students from the Pathway Scholars Program, which is composed of accepted medical students completing a transition year prior to their official matriculation. These groups were selected for both convenience, as other classes were unavailable due to clinical duties, and their minimal baseline knowledge and experience with LPs to reduce confounding based on previous clinical experiences. Informed consent was obtained from all parties. Participants were notified that their participation was voluntary and that they could choose to end their participation at any time during the training. All results were collected via an anonymous and confidential process.

Baseline Testing

All students were provided with a de-identified pre-intervention survey regarding their previous participation in SLIPS, their knowledge of the LP procedure, knowledge of relevant anatomy, and confidence in performing an LP. All participants completed a 15-min orientation led by an emergency medicine physician explaining the LP procedure, then watched a 2-min video created by the SLIPS program. This video explained the correct steps to complete the LP procedure and demonstrated the proper technique that would be evaluated during the study. All students were then moved into a separate testing room and were individually recorded performing a pre-intervention LP. Prior to recording, participants were given a standardized explanation of the study and how the video would be recorded.

Interventions

After baseline testing, students were randomly divided into the VR training group (N = 13) or the SLIPS training group (N = 12). Students in the standard SLIPS training group were taken to a procedural room and were shown the instructional LP video from the original orientation again. They were subsequently subdivided into smaller groups with 1–3 students each and were given 45 min to practice on a task trainer together with SLIPS leaders providing hands-on instruction and feedback in real time at each station.

Students in the VR group were taken to a virtual reality training room. These students were subdivided into smaller groups of 1–3 students each and were given 45 min to practice performing a lumbar puncture on a VR trainer. VR support staff from Arivis were utilized to orient students to the VR equipment. Participants in the VR arm of the study were also able to watch the other members of their group and see what their partners were viewing via a live stream onto computers from their VR goggles. This room was supervised by an emergency medicine physician and the VR support staff.

Post-Intervention Survey and Assessment

After the interventions, all participants were individually taken to the same procedural room and were asked to perform an LP on a synthetic task trainer without any assistance or guidance. Each encounter was recorded on video for review and analysis. All students were instructed to complete a de-identified post-intervention survey regarding the change in their knowledge and confidence in performing the LP procedure, as well as the degree to which their specific intervention was beneficial to their learning.

Performance Evaluation

The video recordings of the pre-intervention and post-intervention LP attempts were reviewed independently by three trained SLIPS leaders, and scored based on an objective Critical Actions Checklist (CAC) (Table 1). The CAC was prepared for this study by the Director of Simulation Curriculum at the UACOM-P. It included the elements necessary for a successful LP, ranging from supply preparation and sterile technique, to performance of the LP procedure itself. Students were given 1 point for adequately completing a step and 0 points for not completing a step, or completing it incorrectly. The time required for students to perform the LP and the total number of times a student punctured the LP task trainer with the needle were both recorded. The data obtained from video scoring was analyzed with interrater reliability calculations using the kappa statistic with the goal baseline of agreement set above 0.90. An aggregate score of the completed CAC items was tallied to evaluate overall intervention effectiveness.

Table 1 Critical Action Checklist
Full size table

Statistical Analysis

Baseline performance measures based on the CAC were compared between groups using Fisher’s exact tests. Changes between pre-intervention and post-intervention performances were evaluated using paired McNemar’s test or exact binomial tests as appropriate. In order to evaluate both interventions together, changes in aggregate score, time to completion, and needle removal were evaluated using Wilcoxon signed rank test. Differences between the SLIPS group and the VR group were evaluated using analysis of covariance adjusted for baseline performance measures. P values less than 0.05 were considered statistically significant and all analyses were completed using SAS 9.4 and STATA 14.

Results

Overall, 25 students participated in this simulation and VR research study. This included 16 first-year medical students and 9 Pathway Scholars Program students. Both cohorts were evenly matched in regard to level of training and prior experience with simulation and VR. All the scores generated from the CAC were subjected to interrater reliability calculations via the kappa statistic with the pre-intervention total points being 0.97 and the post-intervention total points being 0.96.

At pre-intervention, the students were most successful at advancing the needle slowly (100% in both groups) and least successful at maintaining a sterile field (0% in the SLIPS group,15% in the VR group). At baseline, the two groups did not show statistical difference in any measurable outcome, except that 4 (30.8%) students in the VR group advanced without needle adjustment compared to 0 (0%) students in the SLIPS group (Table 2).

Table 2 Baseline demographics and pre-intervention LP assessment
Full size table

Following the intervention, both groups demonstrated improvement in their procedural proficiency in different areas. The SLIPS group improved over the VR group in a variety of metrics including maintaining a sterile field (83% vs. 30%) and correctly preparing the collection vials (92% vs. 15%). Both groups improved in ensuring the needle bevel was oriented; however, only the VR group reached statistical significance (Table 3). The SLIPS group reduced their time to completion by an average of 46.8 s while the VR group reduced their time to completion by an average of 12.8 s (Table 4). Both groups had improvement in the number of students who successfully obtained cerebrospinal fluid (CSF) post-intervention, as the SLIPS group had 92% of participants successful and the VR group had 62% of participants successful. When assessing the intervention as a whole, both groups showed improvement in aggregate score (P < 0.001) and time required to complete (p = 0.002) the lumbar puncture (Table 4). In head to head comparisons, the SLIPS group increased their aggregate score by 3.49 points and decreased their time to completion by 34 s over the VR group when controlling for baseline measures.

Table 3 Pre- and post-intervention assessments of LP proficiency
Full size table
Table 4 Comparison of SLIPS and VR interventions adjusted for baseline measures
Full size table

Prior to interventions, students rated a median score of 1 out of 5 in confidence and a median score of 2 out of 5 on both their knowledge of LP steps as well as its clinical use (Table 5). Knowledge and confidence scores did not statistically vary between SLIPs and VR groups in the pre-intervention analysis. Following interventions, both groups saw significant increases in their knowledge and confidence scores. Specifically, mean confidence on clinical use and steps of LP increased by 1 point and 1.72 points, respectively, while mean confidence scores rose by 1.4 points (Table 6). These increases in knowledge and confidence did not significantly vary between the SLIPS and VR groups.

Table 5 Baseline knowledge and confidence of lumbar punctures
Full size table
Table 6 Pre- and post-intervention assessments of knowledge and confidence of LPs
Full size table

Procedural parameters pre-intervention and post-intervention were tested as an objective measure of knowledge. At pre-intervention, 19 (76%) identified the correct spinal level, and 20 (80%) identified the iliac crest as the appropriate anatomical landmark. Following intervention, all 25 identified the correct spinal level and the iliac crest as the appropriate anatomical landmark.

Discussion

The main purpose of this study was to investigate the feasibility and utility of these two innovative, yet mostly unevaluated, modalities as methods of instruction for the LP procedure. Our principal finding was that both VR and SLIPS training modalities improved students’ proficiency, decreased the time required to complete the procedure, and increased their ability to perform the procedure and obtain CSF. The SLIPS group showed higher performance post-intervention in completing the sequential tasks leading up to needle insertion, such as maintaining a sterile field and opening specimen vials. In addition, they had higher aggregate total objective scores and lower time required for procedure completion versus the VR group. While these findings may show some advantage to hands-on training such as that provided by SLIPS, it also shows VR training to be a feasible modality for learning LP procedures. This is consistent with systematic reviews that have found that simulation significantly improves medical skill acquisition and is superior to other forms of teaching as it is a performance-based method of learning [25].

Although both groups demonstrated significant improvement following their educational interventions, the students who had additional hands-on practice and live facilitator guidance on the actual simulation models performed slightly better than their VR colleagues. This performance disparity may be due to the difference in the educational focus of each modality. Task trainers allow learners to practice holding the spinal needle, receiving tactile feedback from anatomic landmarks, and practicing sterile technique. Learners utilizing VR are able to visualize 3D internal structures and receive haptic feedback for incorrect technique, but are unable to palpate patient external anatomy and practice sequential procedure technique. This difference likely accounted for the longer time to completion and lower aggregate score compared to the SLIPS group. The findings suggest that both VR training and deliberate practice on traditional task trainers have merit and are useful for increasing educational development. When staffing shortages or budget and resource constrictions limit personalized, in-person instruction, VR can be utilized to help learners improve technique with deliberate practice.

Students learn most effectively when they are engaged and feel confident within their abilities [26, 27]. Students’ confidence and knowledge in lumbar puncture were significantly improved in both groups after the intervention. Recent studies have suggested that many residents do not feel adequately prepared to approach clinical skills due to insufficient exposure or hours of practice [15, 28] and lack of confidence in the traditional “see one, do one, teach one” approach to procedural learning [4, 29, 30]. VR and simulation-based learning address this issue by providing tactile experience and feedback not obtained through any other observational or classroom-styled instructions. This is all completed without jeopardizing the health of real patients during training, which is another well-documented benefit of simulation [31]. We suggest that increasing students’ confidence can most readily be achieved by allowing students to practice and observe their own success over time, in conjunction with real-time verbal feedback from trained facilitators.

It is important to note that there are several limitations to the study. First, the students randomized to the SLIPS group had the advantage of having additional time and practice on the same task trainers that were used in the post-intervention testing for both groups, and watched the instructional video an additional time compared to the VR group. Future studies should aim to have a different simulation task trainer used for final assessment of the two groups and attempt to eliminate potential confounding supplementary material. The SLIPS group also received much more targeted instruction regarding the sequential procedural elements that were eventually tracked via the CAC. Given this confounding variable, the primary focus of the study was not comparing the aggregate scores between groups, but rather the time required for completion following picking up the LP needle, and whether students were successful in obtaining CSF for collection.

In addition, neither group learned the LP procedure from an experienced physician. It is a novel application of students to teach procedures, since they are cheaper than experienced clinicians, more readily available, and likely less intimidating to their peers versus attending physicians. Near-peer learning has been shown to be effective and feasible for teaching procedural skills [18,19,20, 32]. However, students have less experience, and therefore much less technical skill than trained physicians. Future validation studies could examine the difference in aggregate scores in clinical skills trainings for students that were led by experienced physicians versus trained peers.

Finally, our sample size for this study was small due to difficulties in recruitment. Future studies should have a larger representation of the student body with the potential for further analysis of differing levels of experience in the targeted learners.

Some medical programs have contrasted the difference between high- and low-fidelity simulators [25], while others are looking into expanding the proportion of VR use [15, 16]. Based on the results of this study, the best approach could be to incorporate both modalities. The ability to practice full procedures, including sterile technique and equipment preparation with a task trainer, complimented by the power of visualizing internal structures with VR will enable all types of learners to enhance their education in manners previously unavailable. We expect combining both modalities would lead to a more comprehensive skill set and improved competency in performing clinical procedures.

Conclusion

Student-led simulation can be used to help medical students gain confidence and competence in performing procedures such as lumbar punctures. Virtual reality modules can enhance spatial recognition and anatomic visualization and provide guided practice without the need for personalized instruction. Practice with simulated task trainers under near-peer direction and virtual reality programs are both feasible options for medical education skills training, well-received by students, and can be used to engage learners and accelerate proficiency in performing procedures. Future studies are recommended to evaluate how these skills translate into the clinical arena and the improvement of patient care.