Percutaneous Spinal Fixation Simulation With Virtual Reality and Haptics

Luciano, Cristian J. PhD*; Banerjee, P. Pat PhD*,‡,§,¶; Sorenson, Jeffery M. MD; Foley, Kevin T. MD; Ansari, Sameer A. MD, PhD#; Rizzi, Silvio*; Germanwala, Anand V. MD**; Kranzler, Leonard MD; Chittiboina, Prashant MD, MPH‡‡; Roitberg, Ben Z. MD¶,§§

doi: 10.1227/NEU.0b013e3182750a8d
Surgical Exposure/Microsurgery

BACKGROUND: In this study, we evaluated the use of a part-task simulator with 3-dimensional and haptic feedback as a training tool for percutaneous spinal needle placement.

OBJECTIVE: To evaluate the learning effectiveness in terms of entry point/target point accuracy of percutaneous spinal needle placement on a high-performance augmented-reality and haptic technology workstation with the ability to control the duration of computer-simulated fluoroscopic exposure, thereby simulating an actual situation.

METHODS: Sixty-three fellows and residents performed needle placement on the simulator. A virtual needle was percutaneously inserted into a virtual patient’s thoracic spine derived from an actual patient computed tomography data set.

RESULTS: Ten of 126 needle placement attempts by 63 participants ended in failure for a failure rate of 7.93%. From all 126 needle insertions, the average error (15.69 vs 13.91), average fluoroscopy exposure (4.6 vs 3.92), and average individual performance score (32.39 vs 30.71) improved from the first to the second attempt. Performance accuracy yielded P = .04 from a 2-sample t test in which the rejected null hypothesis assumes no improvement in performance accuracy from the first to second attempt in the test session.

CONCLUSION: The experiments showed evidence (P = .04) of performance accuracy improvement from the first to the second percutaneous needle placement attempt. This result, combined with previous learning retention and/or face validity results of using the simulator for open thoracic pedicle screw placement and ventriculostomy catheter placement, supports the efficacy of augmented reality and haptics simulation as a learning tool.

ABBREVIATION: AANS, American Association of Neurological Surgeons

Departments of *Mechanical and Industrial Engineering

Bioengineering, and

§Computer Science, College of Engineering, University of Illinois at Chicago, Chicago, Illinois

Divisions of Neurosurgery and

§§Radiology, University of Chicago, Chicago, Illinois

Medical Education and Research Institute, Memphis, Tennessee

#Department of Radiology, Neurology, and Neurosurgery, Northwestern University, Chicago, Illinois

**Department of Neurosurgery, The Johns Hopkins School of Medicine, Baltimore, Maryland

‡‡Department of Neurosurgery, Louisiana State University, Shreveport, Louisiana

Correspondence: P. Pat Banerjee, PhD, Departments of Mechanical and Industrial Engineering, University of Illinois at Chicago, 2039 ERF, M/C 251, 842 W, Chicago, IL 60607. E-mail:

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s Web site (

Received May 05, 2012

Accepted August 23, 2012

Article Outline

At the 2010 annual meeting of the American Association of Neurological Surgeons (AANS), the Young Neurosurgeons Committee continued its annual tradition of organizing a surgical competition using emerging simulators for residents and fellows in the exhibit hall. Percutaneous spinal needle placement was one of the techniques tested, and the results of this testing are reported here. To increase the number of subjects, the same experiment was repeated in January 2011 at the Chicago Review Course on Neurological Surgery, an annual event since 1974 that draws surgeons and residents from all over the world ( The performance of 63 fellows and residents was evaluated for percutaneous spinal needle placement using the head- and hand-tracked high-resolution and high-performance augmented-reality and haptics workstation known as ImmersiveTouch (ImmersiveTouch, Inc). Using a high-definition stereoscopic display and a translucent (half-silvered) mirror, the ImmersiveTouch system1,2 superimposes the 3-dimensional (3-D) projection of a virtual instrument over a haptic device stylus, providing force feedback as the trainee performs the virtual surgery on a 3-D model obtained from computed tomography (CT) and/or magnetic resonance imaging of a real patient. An electromagnetic tracking system continuously tracks the position and orientation of the user’s head, achieving perfect graphics/haptics collocation as the user moves his/her head, providing the illusion that the surgery is actually performed in a real patient. The system has been evaluated for applications such as ventriculostomy,3-6 ventriculoperitoneal shunt placement,7 and open thoracic pedicle screw placement.8

Unlike open spinal fixation, in which the surgeon is able to see the bone anatomy, percutaneous spinal fixation requires the surgeon to rely mainly on tactile feedback and fluoroscopy images to guide the proper insertion of the needle through the skin and muscle into the pedicles. Percutaneous spinal fixation needs a different set of skills compared with the equivalent open surgical procedure.

The experiment presented by Luciano et al8 focused on open spinal fixation. It was conducted to study whether the trainees are able to understand the internal spinal structure involved in the procedure, looking at both the 3-D model of the exposed spine and multiple views (anteroposterior and lateral) of the fluoroscopy images and CT images while they insert the needle into the pedicles. During the practice session, the participants were requested to perform multiple insertions with the assistance of fluoroscopy and CT images. After the practice session, the CT and fluoroscopy images were hidden; therefore, the participants had to remember the skills learned during the practice session and to perform insertions based only on the appearance of the 3-D anatomy and their tactile sensations but with no image guidance at all.

Unlike the previous experiment, the goal of the study reported here (focused on percutaneous spinal fixation) was to determine whether the trainees are able to perform needle insertions with limited image guidance assistance (but without seeing the 3-D spine model), trying to focus on their haptic perception to improve their accuracy while minimizing the amount of radiation exposure applied to the patient.

Spinal fixation presents high risks to the patients when performed by inexperienced neurosurgery residents in their early levels of training; therefore, there is a prominent need for such simulation. The ImmersiveTouch system offers a number of convenient options for percutaneous needle placement training, allowing the residents to better understand the spinal structure involved in the procedure and then to practice in one of the most critical steps in spinal fixation. Other important steps of the procedure, including placement of rod fixation, are easier to teach and therefore are excluded from the simulation. As shown in Figure 1, the user has the option of monitoring the trajectory of the needle by simulated anteroposterior, transverse, and lateral fluoroscopic views as a means of image guidance. These views can be requested by pressing a foot pedal. The views are available only when the pedal is pressed, and the duration of the foot pressing the pedal is recorded to determine the amount of fluoroscopic exposure.

The user also has the option to adjust a number of visual and haptic parameters by using a pop-up menu as shown in the bottom right of Figure 2. As an example, Figure 2 shows the same view as Figure 1 but with the skin of the patient shown in semitransparent mode to clearly show the trajectory of the needle. Depending on the needs and the skill level of the user, the system can be adjusted to aid the learning process.

Back to Top | Article Outline


The objective is to experimentally determine learning effectiveness in terms of entry point/target point accuracy of percutaneous spinal needle placement. The user first undergoes training through a practice session. Subsequently, the user is asked to repeat the same task that was practiced in the test session. Two attempts are recorded in the test session. The learning effectiveness evaluates comparative performance of the first and second attempts.

Back to Top | Article Outline


Selection Criteria

Through open solicitations, a total of 63 fellows and residents were enrolled as participants. One group was enrolled by the Young Neurosurgeons Committee at the 2010 AANS annual meeting, and the other group was enrolled by Leonard Kranzler at the 2011 Chicago Review Course on Neurological Surgery. The ImmersiveTouch system used for the experiment was provided by the University of Illinois at Chicago and partially supported by ImmersiveTouch, Inc. A virtual 3-D volume of a human spine was created using a CT from a patient at the University of Illinois at Chicago Medical Center. The data were presegmented and assembled from a CT DICOM (digital imaging and communications in medicine) data set after the removal of all identifying personal data. The 3-D polygonal isosurfaces corresponding to the skin and underlying spinal column were extracted.

Back to Top | Article Outline


The participants were given approximately 5 minutes to practice on any pedicle of their choice from among the 6 pedicles: the left and right T9, T10, and T11. During the practice session, the participants are taught the importance of needle location accuracy while requesting minimal real-time computer-simulated fluoroscopic images in anteroposterior and laterals views, as shown in Figure 1. The fluoroscopic images can be requested by pressing a foot pedal. When the user is satisfied with the final location of the needle, a second foot pedal is pressed to freeze the result. The computer shows the needle location in both fluoroscopy views (anteroposterior and lateral) and a transverse view of the CT image (typically obtained after the procedure to verify its outcome). This permitted the users to track all the details of their percutaneous targeting primarily by haptic feedback and periodic image feedback. After practice, the test session consisted of using the skills learned during practice. The user can then use the cutting tool while moving and rolling the wrist and rotating the head to visualize the exact location of the screw and to correlate the experience with technique (Figure 3). For each participant, the final position and orientation of the screw were recorded by the computer.

While pushing the needle through the spine, the haptic stylus can be moved along a linear trajectory defined according to its orientation. A reactionary force is applied by the haptic device as the user deviates from that linear trajectory. This is intended to simulate a firm-feeling needle placement that is similar to the tactile sensation experienced during surgery. If trajectory is to be changed, then the user has to take the needle out of the pedicle by backtracking on the same linear trajectory before reinserting the needle along a different linear trajectory (see Video 1, Supplemental Digital Content 1,, which provides a visual highlight of the percutaneous spinal needle placement simulation).

Back to Top | Article Outline

Measurement Technique

Performance data are collected for 2 successive needle placements. Because performance error and radiation exposure are equally important to identify whether the percutaneous spinal fixation has been successful, 2 parameters were measured: accuracy in terms of average euclidean distance from predefined entry and target points and the duration of fluoroscopic exposure. A cumulative score was computed by giving equal weight to each of these 2 parameters. The aggregate score is the sum of all individual scores. The lower the score is, the better the performance is.

Back to Top | Article Outline

Performance Accuracy Measurement

During the practice and test sessions, the participants are given a recommended landmark on each pedicle from which to start the needle insertion and another landmark at which to stop the insertion process. The accuracy score (or performance error) is based on the euclidean distance in millimeters from these targets. If the screw is placed outside the spinal body, it carries a penalty of 200 (which is outside of the normal distance range) on the evaluation score to indicate failure of the procedure.

Back to Top | Article Outline

Fluoroscopic Exposure Measurement

The amount of fluoroscopic exposure is simulated by the duration of foot pedal application in milliseconds. On application of the foot pedal, the current location of the needle is shown in the anteroposterior and lateral views.

Back to Top | Article Outline

Failure Rate Measurement

The failure rate is the ratio of the number of failures to the total number of attempts.

Back to Top | Article Outline


Ten of 126 needle placement attempts by 63 participants ended in failure for a failure rate of 7.93%. Thirty-three of the 63 participants improved, 28 worsened, and 2 maintained their performance score from the first to the second attempt. From all 126 needle insertions, the average error (15.69 vs 13.91), average fluoroscopy exposure (4.6 vs 3.92), and average individual performance score (32.39 vs 30.71) improved from the first to the second attempt. For performance accuracy, an aggregate euclidean distance deviation from entry landmark on the pedicle and a similar deviation from the target landmark in the vertebral body yielded P = .04 from a 2-sample t test in which the rejected null hypothesis assumes no improvement in performance accuracy from the practice to the test sessions and the alternative hypothesis assumes an improvement.

In terms of accuracy, the experiment reported a slightly flattened distribution of the performance error from the first to the second attempt (Figure 4).

Because percutaneous spinal fixation depends mainly on the ability to interpret fluoroscopy images while maintaining the amount of radiation to a minimum to increase patient safety, it is interesting to analyze whether the simulator is able to reduce the amount of fluoroscopy exposure applied by the trainees during the procedure. A comparison of the results obtained between first and second attempts showed that the participants required a significantly lower amount of fluoroscopy shots after a short practice on the simulator (Figure 5).

Even though the number of needle placement failures during the first and second attempts was the same (5 occurrences), the analysis of the distribution of the final scores, including both performance error and fluoroscopy exposure, showed that the participants were able to improve the outcomes of the virtual surgery during their second attempt (Figure 6).

To assess the impact of duration of requested fluoroscopic image guidance and performance accuracy, the null hypothesis is as follows: “the change in fluoroscopic exposure use does not associate with the change in accuracy achieved.” The results do not show any meaningful correlation between the amount of fluoroscopic image duration request and performance accuracy; hence, the null hypothesis cannot be rejected.

The experiment could not correlate scores with level of training because they were not provided by AANS and the Chicago Review Course organizers to maintain the anonymity of the participants.

Back to Top | Article Outline


The purpose of this study was to evaluate the accuracy of percutaneous spinal needle placement on a head- and hand-tracked high-resolution and high-performance virtual reality and haptic technology workstation. Specifically, we wish to understand the role of virtual reality and haptics simulation in a variety of situations, in this case, percutaneous spinal fixation. The test bed for practicing percutaneous spinal fixation consists of a virtual patient constructed from actual CT scans. A broader goal was to demonstrate the feasibility of using a virtual reality environment as a simulator of spinal neurosurgical procedures by understanding the interplay of various parameters affecting surgical performance such as accuracy of needle placement and amount of fluoroscopic exposure.

The study demonstrated that the ImmersiveTouch system can be adapted to perform various simulations and the ubiquitous nature of “entry point/target point” situations in any invasive procedure. It also demonstrated that computerized fluoroscopic imaging time simulation can be used to help understand the training process.

The accuracy of spinal needle placement achieved by participants using the simulator is comparable to the accuracy reported in a recent retrospective evaluation of such placements.9 Our results demonstrate significant improvement from 1 attempt to the next within the virtual reality environment. We plan future studies with a greater number of attempts to further explore whether a greater number of attempts will result in greater performance improvement and at what rate.

Approaches similar in spirit to ours have been reported in the literature. For example, Braak et al9 reported a real-time 3-D fluoroscopy guidance using cone-beam CT with dedicated needle path planning software as a promising new interventional technique. They assessed the accuracy and feasibility of this technique for use in needle interventions. Their approach allowed the use of fluoroscopy coregistered with a 3-D data set reconstructed from the acquired attenuation information. They projected the calculated trajectory onto the real-time fluoroscopy image. However, our approach encapsulates the primary rationale behind such approaches and captures them within the context of a simulator, which is much easier to use than such clinical approaches and at a fraction of the cost. Braak et al reported fluoroscopy time, accuracy, technical success of the procedures, median procedure time, and complications for 145 interventions on 139 patients. In our simulator, we capture the fluoroscopy time, accuracy, and procedure time. They reported that all interventions were within the predefined 5-mm safety margin and achieved 100% technical success. The median interventional procedure time was 28.5 minutes, and the median fluoroscopy time was 2 minutes 58 seconds. There were minor complications in 6 patients (4.3%) and 1 major complication (0.7%). These statistics are comparable to some of the estimates from our simulator, thereby indicating some degree of face validity.

Similarly, Tam et al10 reported a retrospective review of early clinical experience of 10 consecutive oncology patients who underwent vertebroplasty of 13 vertebral levels with C-arm cone-beam CT with fluoroscopic overlay for needle guidance during vertebroplasty. Procedural data, including vertebral level, approach (transpedicular vs extrapedicular), and access (bilateral vs unilateral), were recorded. Technical success with the overlay technology was assessed on the basis of accuracy, which consisted of 4 measured parameters: distance from target to needle tip, distance from planned path to needle tip, distance from midline to needle tip, and distance from the anterior one-third of the vertebral body to needle tip. Technical success rates were 92% for both distance from planned path and distance from midline to final needle tip, 100% when distance from needle tip to the anterior one-third border of the vertebral body was measured, and 75% when distance from target to needle tip was measured. Once again, our simulator results bear some face validity to some of these results.

A number of studies using anatomy-specific CT-based finite element methods have been conducted to simulate percutaneous spinal fixation11-14 to develop a deeper understanding of which vertebrae have the highest risk of fracture or other severe complications. Similarly, biomechanical simulations, eg, cement leakage in percutaneous vertebroplasty,15 have been conducted to understand the vertebroplasty process better. Haptic feedback has been used in some of these biomechanical simulation studies16,17 because the physician relies on both sight and feel to properly place the bone needle through various tissue types and densities and to help monitor the injection of cement into the vertebra. The human-computer interaction for simulating cement injection in such virtual spine workstation is conceptually similar to our approach; however, the haptics and problem area studied here are different. In the Chui et al16 study, fluoroscopic images were generated from the CT patient volume data and simulated volumetric flow using a time-varying 4-dimensional volume-rendering algorithm. The user’s finger movement was captured by a data glove. Immersion CyberGrasp was used to provide the variable resistance felt during injection by constraining the user’s thumb. However, such haptics approaches are gradually becoming obsolete because of the advent of superior haptics technology such as that used in our approach.

Kobayashi et al18 performed a study to evaluate the accuracy of puncture to the median vertebral body using the unilateral transpedicular approach on percutaneous spinal fixation. They developed a simple puncture simulation method based on the puncture angle determined by preoperative CT. The percutaneous spinal fixation simulator reported here can potentially contribute to validation studies for such approaches.

Back to Top | Article Outline


Although limited features of our simulator were used in controlled validation experiments to reduce variability in data collection, the capabilities of the simulator go beyond what is reported here. We have a state-of-the-art percutaneous spinal needle placement simulator that allows experimentation with any patient CT data set. It uses standard Microsoft Windows platform and common off-the-shelf controls such as mouse, keyboard, foot pedals, Nintendo Wii remote controls for 3-D mouse, and a standard haptic stylus that can hold any physical percutaneous spinal fixation needle. In addition, the user can adjust material properties of the bone, tissues, and skin; view in transparent or semitransparent mode for ease of learning; adjust the fluoroscope from continuous to periodic to occasional use settings; and adjust the user background from novice to intermediate to expert settings. The same simulator also can be adapted to other percutaneous and open surgical8 procedures. The greater realism presented in our technology can lead to more effective learning. The experiments showed evidence (P = .04) of improvement of performance accuracy from the first to the second percutaneous needle placement attempt. The analysis of the average and the distributions of accuracy, fluoroscopy exposure, and performance score during the first and second attempts also showed that the participants improved their outcomes and reduced the risks of complications after a short practice on the simulator. This result, combined with the previously published results8 of using the ImmersiveTouch simulator for open thoracic pedicle screw placement for learning retention from practice session to the test session, supports the efficacy of ImmersiveTouch as a learning tool. Having demonstrated similar efficacy in prior work on ventriculostomy3-6 and pedicle screw simulations,8 ImmersiveTouch continues to evolve as a versatile training tool that can be adapted to a variety of situations as needed.

Back to Top | Article Outline


The research was supported in part by National Institutes of Health National Institute of Biomedical Imaging and Bioengineering grant 1R21EB007650-01A1. Presented at the 2010 AANS annual meeting, this study was sponsored by the AANS Young Neurosurgeons Committee, which does not claim the superiority of the ImmersiveTouch system over another system. The ImmersiveTouch technology has been licensed to ImmersiveTouch, Inc, by the University of Illinois at Chicago. Dr Banerjee is a part-time employee of and owns stock in ImmersiveTouch, Inc. Dr Luciano is a part-time employee of ImmersiveTouch, Inc. The other authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.

Back to Top | Article Outline


We would like to acknowledge the help of Naga Dharmavaram and FeiFei Liu in organizing the data and in running some statistical tests.

Back to Top | Article Outline


1. Banerjee PP, Luciano C, Florea L, et al., inventors. Compact haptic and augmented virtual reality system. US patent 7812815, 2010 assigned to University of Illinois.
2. Luciano C, Banerjee P, Florea L, Dawe G. Design of the ImmersiveTouch: a high-performance haptic augmented virtual reality system. In: Proceedings from the International Conference on Human-Computer Interaction (HCI); July 2005; Las Vegas, NV.
3. Banerjee PP, Yudkowsky R, Lemole M, Charbel F, Luciano C. Using a high-fidelity virtual reality and haptics-based simulation to determine the “learning curve” of neurosurgery resident's surgical skills. Journal of Simulation in Healthcare. 2007;2(2):145.
4. Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY. Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg. 2007;107(3):515–521.
5. Lemole GM Jr, Banerjee PP, Luciano C, Neckrysh S, Charbel FT. Virtual reality in neurosurgical education: part-task ventriculostomy simulation with dynamic visual and haptic feedback. Neurosurgery. 2007;60(1):142–149.
6. Lemole M, Banerjee PP, Luciano C, Charbel F, Oh M. Virtual ventriculostomy with “shifted ventricle”: neurosurgery resident surgical skill assessment using a high-fidelity haptic/graphics virtual reality simulator. Neurol Res. 2009;31(4):430–431.
7. Oh M, Banerjee PP, Zhang K, et al.. Ventriculoperitoneal shunt technique assessment using a high-fidelity haptic/graphics virtual reality simulator. In: Proceedings from the 23rd International Computer Assisted Radiology and Surgery (CARS) Congress; 2009; Berlin, Germany.
8. Luciano CJ, Banerjee PP, Bellotte B, et al.. Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery. 2011;69(1 suppl operative):ons14–ons19.
9. Braak SJ, van Strijen MJ, van Leersum M, van Es HW, van Heesewijk JP. Real-time 3D fluoroscopy guidance during needle interventions: technique, accuracy, and feasibility. AJR Am J Roentgenol. 2010;194(5):W445–W451.
10. Tam AL, Mohamed A, Pfister M, et al.. C-arm cone beam computed tomography needle path overlay for fluoroscopic guided vertebroplasty. Spine. 2010;35(10):1095–1099.
11. Chae SW, Kang HD, Lee MK, Lee TS, Park JY. The effect of vertebral material description during vertebroplasty. Proc Inst Mech Eng H. 2010;224(1):87–95.
12. Wolfram U, Wilke HJ, Zysset PK. Valid micro finite element models of vertebral trabecular bone can be obtained using tissue properties measured with nanoindentation under wet conditions. J Biomech. 2010;43(9):1731–1737.
13. Wijayathunga VN, Jones AC, Oakland RJ, Furtado NR, Hall RM, Wilcox RK. Development of specimen-specific finite element models of human vertebrae for the analysis of vertebroplasty. Proc Inst Mech Eng H. 2008;222(2):221–228.
14. Kosmopoulos V, Keller TS. Damage-based finite-element vertebroplasty simulations. Eur Spine J. 2004;13(7):617–625.
15. Gisep A, Boger A. Injection biomechanics of in vitro simulated vertebroplasty: correlation of injection force and cement viscosity. Biomed Mater Eng. 2009;19(6):415–420.
16. Chui CK, Teo J, Wang Z, et al.. Integrative haptic and visual interaction for simulation of PMMA injection during vertebroplasty. Stud Health Technol Inform. 2006;119:96–98.
17. Lian Z, Chui CK, Teoh SH. A biomechanical model for real-time simulation of PMMA injection with haptics. Comput Biol Med. 2008;38(3):304–312.
18. Kobayashi K, Takizawa K, Koyama M, Yoshimatsu M, Sakaino S, Nakajima Y. Unilateral transpedicular percutaneous vertebroplasty using puncture simulation. Radiat Med. 2006;24(3):187–194.

Haptics; Neurosurgical simulation; Percutaneous spinal fixation; Virtual reality

Supplemental Digital Content

Back to Top | Article Outline
Copyright © by the Congress of Neurological Surgeons