BACKGROUND: In the past decade, surgery planning has changed significantly. The main reason is the improvements in computer graphical rendering power and display technology, which turned the plain graphics of the mid-1990s into interactive stereoscopic objects.
OBJECTIVE: To report our experiences with 2 virtual reality systems used for planning neurosurgical operations.
METHODS: A series of 208 operations were planned with the Dextroscope (Bracco AMT, Singapore) requiring the use of liquid crystal display shutter glasses. The participating neurosurgeons answered a questionnaire after the planning procedure and postoperatively. In a second prospective series of 33 patients, we used an autostereoscopic monitor system (MD20-3-D; Setred SA, Sweden) to plan intracranial operations. A questionnaire regarding the value of surgery planning was answered preoperatively and postoperatively.
RESULTS: The Dextroscope could be integrated into daily surgical routine. Surgeons regarded their understanding of the pathoanatomical situation as improved, leading to enhanced intraoperative orientation and confidence compared with conventional planning. The autostereoscopic Setred system was regarded as helpful in establishing the surgical strategy and analyzing the pathoanatomical situation compared with conventional planning. Both systems were perceived as a backup in case of failure of the standard navigation system.
CONCLUSION: Improvement of display and interaction techniques adds to the realism of the planning process and enables precise structural understanding preoperatively. This minimizes intraoperative guesswork and exploratory dissection. Autostereoscopic display techniques will further increase the value and acceptance of 3-dimensional planning and intraoperative navigation.
ABBREVIATIONS: CTA, computed tomographic angiography
DICOM, digital imaging and communications in medicine
LCD, liquid crystal display
MRA, magnetic resonance angiography
VR, virtual reality
*Neurochirurgische Klinik, Universitätsmedizin der Johannes Gutenberg-Universität Mainz, Mainz, Germany
‡Department of Neurosurgery, Klinik Hirslanden, Zürich, Switzerland; and Department of Neurosurgery, University Hospital Mainz, Mainz, Germany
Correspondence: Axel Stadie, MD, Neurochirurgische Klinik, Universitätsmedizin der Johannes Gutenberg-Universität Mainz, Langenbeckstraße 1, Mainz, Germany. E-mail: Axel.Stadie@me.com
Received June 06, 2012
Accepted August 09, 2012
The primary goal of planning a surgical intervention is anticipating the intraoperative findings and deciding in advance on a clear surgical strategy. Having vague ideas about the intraoperative situation and the surgical strategy may lead to unfavorable outcome.
Nowadays, neurosurgeons have a wide range of options to assess the individual pathoanatomical situation of each patient preoperatively. Apart from the patient’s history and clinical examination, surgical planning relies on the study of preoperative radiological data such as magnetic resonance imaging (MRI), computed tomography (CT), digital x-ray angiography, and magnetic resonance angiography (MRA) or CT angiography (CTA). Apart from being presented in 2-dimensional (2-D) imaging series, this information can be shown as volumetric data, including 3-dimensional (3-D) user interfaces for 3-D manipulations. All these modalities provide detailed information about different structures. For surgical approach planning, the neurosurgeon has to integrate this variety of information mentally into a 3-D concept. The difficulty in creating this mental representation increases with the complexity of spatial relationships of intracranial structures and is complicated by the fact that the imaging series vary in slice thickness, scale, and patient orientation.1 To make this task easier, in the 1980s, several research groups started providing 3-D reconstructions of imaging data. Starting with reconstructions of CT data that were presented as static screen photographs from different angles,2-4 MRI data were implemented in surgery planning beginning in the early 1990s,1 and in the years to follow, various computer programs capable of generating 3-D representations of tomographic imaging data for surgical planning were developed.5-7 In the 1990s, several groups evaluated the usefulness of a variety of neurosurgical planning programs that combined real-time interaction with 3-D imaging data to various degrees.5,8-10 However, common to all these planning systems was the fact that interaction with the increasingly sophisticated 3-D data sets was still being achieved in a rather nonintuitive 2-D fashion: by moving a screen-bound cursor with a mouse. However, the 3-dimensionality of a neurosurgical procedure should be reflected during the planning. Hence, the ideal neurosurgical planning environment should present a 3-D computer-generated reconstruction of the imaging data in a 3-D stereoscopically perceivable space and allow interaction in a natural, easy, and direct manner.1 Following this idea, a workstation called Dextroscope emerged from the research field of virtual reality (VR). It allows the display of multimodal patient-specific models in a 3-D workspace. With this system, one can perceive the 3-D graphical objects stereoscopically wearing liquid crystal display (LCD) shutter or polarizing glasses and interact with them using virtual volume exploration tools.1,11 Because wearing active or polarizing glasses imposes some degree of discomfort, research has been focusing on the development of display systems that allow perceiving a 3-D image simply with the naked eyes. Reports on the use of these autostereoscopy systems in medicine are still scarce. Abildgaard et al12 reported positive results in their evaluation of whether the visualization of static 3-D models from intracranial time-of-flight MRA was improved with the use of an autostereoscopic display compared with a 3-D rendering on a 2-D screen.
We present our experiences using the Dextroscope for neurosurgical planning over a period of 6 years and report our first results using the Setred Planning station incorporating an autostereoscopic display.
MATERIALS AND METHODS
Surgeons planned all their interventions of this series using the available preoperative images (CT, CTA, MRI, MRA, positron emission tomography [PET] scans) before using the VR system. The images were stored on a picture archiving and communication system and were viewed on 2-D flat screens. The main surgeon and 1 or 2 assistants planned all cases 1 to 2 days before the actual surgery. All in-house imaging series were acquired 1 to 14 days preoperatively and stored on the system server in digital imaging and communications in medicine (DICOM) 3 format. CT images were usually presented as axial images. MRI data were preoperatively typically shown as axial, coronal, and sagittal images. The 3-D reconstructions of CT images or MRIs were generated on demand by the Department of Neuroradiology. Reconstructions were either additional planes (eg, secondary reconstructions of sagittal planes from axial scans) or pseudo--3-D reconstructions (eg, depiction of the circulus arteriosus cerebri from MRA data). The planning tools of neuronavigation system have not been used in this series. Surgeons did not have access to stereoscopic monitors other than the Dextroscope.
VR Image Processing
The hardware, software, and functionality of the Dextroscope have been described in detail.1,11,13,14 The images transferred into the Dextroscope were displayed by 3-D volume rendering. In case of multimodality visualization, the CT and MRI data were automatically coregistered and displayed as one. The segmentation of structures that were demarcated by their outstanding intensity on the gray scale (such as contrast-enhancing tumors, vessels, ventricles, and bone or skin surface) is achieved by semiautomatic adjustments of the color lookup tables or interactive region-growing algorithms. Other structures were generated with the help of manual outlining tools. The resulting multimodal and multisegmented 3-D data set is typically a mix of volume-rendered and polygonal surface structures, displayed in stereo and shaded to various degrees. We measured the time needed from beginning of the data import into the Dextroscope until the beginning of surgery planning. This period of time was documented for each of the 208 patients as time needed to build the virtual patient model.
After creating the virtual patient model, the individual pathoanatomy of the patient was explored with the Dextroscope. Different approaches were simulated. Simulating different head positions made it possible to demonstrate various intraoperative views to explore the spatial relationships of the individual surgical corridors.
All planning procedures were evaluated preoperatively and postoperatively by the participating surgeons using a questionnaire. We continued writing a short report about each planning session, including imaging data used for planning, the time spent for data preparation, and the software tools used for segmentation and planning. We also recorded the specific advantages or shortcomings of the system related to each individual case of surgical planning, and the primary surgeon continued to answer a short questionnaire. This questionnaire was divided into a presurgical and a postsurgical part; 20 different surgeons completed these questionnaires. The presurgical part consisted of 3 questions concerning the degree of improvement in spatial understanding of the surgical target and related structures, the degree of improvement in planning the size and location of the craniotomy (compared with conventional planning), and whether the Dextroscope planning session led to a change in surgical approach compared with planning the approach with conventional 2-D images. The postsurgical part consisted also of 3 questions on the degree of intra-operative improvement in spatial orientation after the use of the Dextroscope for planning, the degree of improvement in overall intraoperative confidence, and whether the chosen surgical plan (choice of craniotomy location and surgical corridor) had proven correct. The first 2 questions of both parts of the questionnaire were multiple choice questions requiring the surgeons to indicate whether they had seen “no improvement,” “little improvement,” “good improvement,” or “very good improvement.” The third question in both parts required a “yes” or “no.” Additionally, free responses of the surgeons were documented if they were given.
The technology of the Setred system (Setred AS, Oslo, Norway; Figure 1) has been described briefly by Abildgaard and colleagues.12 The system consists of a personal computer, the MD20-3-D autostereoscopic monitor, and Samurai software controlling the monitor. A personal computer performs the corresponding renderings for different viewing angles. The monitor screen size is 20 in, and the effective refresh rate is approximately 100 Hz. In autostereoscopic mode, the resolution is 1024 × 768 for each viewing position. The MD20-3-D is a “scanning slit display” that works according to the principles illustrated in Figures 2 and 3. A person looking through a window will see different parts of an object with each eye. In almost the same manner, an observer looking through a narrow vertical slit will see distinct parts of the item behind the slit. These slight differences of viewing angles result in stereoscopic depth perception. In the MD20-3-D the images are projected by a high-frame-rate (around 3-4 kHz) projector onto a diffusive screen. There are 288 vertical slits in front of the diffusive screen. The slits are controlled electronically by an LCD so that each slit is either transparent or nontransparent. Several slits scan the screen rapidly (repetition rate at about 100 Hz for a single slit, resulting in a new slit position every 270 microseconds), and they are synchronized with the 2-D display that reproduces the image that would be seen through the open slit. Because of the fast scanning speed, the viewer will not perceive the panel with slits but rather a transparent glass panel. Thus, in 1 full scan cycle, the observer will have seen all parts of the 2-D display. Because the observer’s eyes are in different physical positions, the image that each eye perceives is slightly different. This is the basis for the stereoscopic effect of the vertical slit scan display. The system provides separate views for 37 horizontal directions, covering a total horizontal viewing angle from 20° to 30°. The number of horizontal views or zones can be configured to up to 200 views. The 3-D is not limited to a “sweet spot” (the zone in which comfortable 3-D viewing is possible), as is typical for other auto 3-D displays. When moving horizontally in front of the display, clear 3-D perception is possible in a range of 90°, with the image only slightly blurred when positioned between 2 zones. Head or eye tracking is not necessary, and the number of people who can view a stereo object at the same time is technically not limited.
Setred Hardware Setup
A personal computer equipped with off-the-shelf graphic cards controls the autostereoscopic monitor. In our setup, we used a personal computer (Dell, Austin, Texas) with 2.81 GHz and 8 GB of RAM (random-access memory) running Windows XP SP2 (Microsoft, Redmond, Washington), and 3 graphic cards (NVIDIA, Santa Clara, California). A second LCD monitor is positioned adjacent to the autostereoscopic monitor to display the Samurai software.
Setred Data Preparation
The MRI or CT data are transferred to the MD20- 3-D in DICOM format and then displayed as a 3-D stereoscopic object. Either a USB stick or an Ethernet network connection can be used.
The Samurai software generating the volumes and controlling the autostereoscopic monitor was developed by Setred AS. It consists of a data-importing interface allowing the upload of different imaging series modalities (MRI or CT); however, it is possible to display only the MRI or the CT, 1 at a time. After the 3-D model is created, it can be modified by assigning colors according to the gray scale. In this way, for example, a tumor can be displayed in green, the surrounding brain tissue in brown, and the ventricles in blue (compare illustrative case 2). The colors can be defined manually or via adjustable presets.
Surgeons planned all their interventions of this series using the available preoperative images (CT, CTA, MRI, MRA, PET scans) before using the MD20-3-D. The images were stored on a picture archiving and communication system and were viewed on 2-D flat screens. Surgeons used secondary reconstruction software in the Department of Neuroradiology on demand; however, we have not collected data on how surgeons prepared their interventions conventionally.
Setred Surgical Planning
After the virtual patient model has been created, the user is free to explore it using the autostereoscopic monitor. The model is controlled by a regular computer mouse. Right and left clicks enable rotation and translation, and the scrolling wheel of the mouse allows object magnification and adjustment of the focal point along the z axis into the depth of the volume. Crop tools allow cutting the model in the x, y, or z axis. Additionally, there are 2 sphere tools. The first sphere tool displays the image information inside the sphere, and the second sphere tool acts like a virtual drill removing all image information inside the sphere. Both sphere tools can be scaled and moved along the z axis with the scrolling wheel of the mouse. A simple tool to measure a straight line is also available.
Surgeons using the MD20-3-D in this series were asked to answer a questionnaire after planning their intervention and after surgery. The answers were given in free form.
Neurosurgeons had to define preoperatively the expectations to which autostereoscopic planning was applied. They also had to evaluate postoperatively whether these expectations were fulfilled.
Our results after planning 208 neurosurgical operations with the Dextroscope confirm our initial results published in 2008. Table 1 gives an overview of the pathologies planned with the Dextroscope. We found that with growing experience using the Dextroscope, less time was required to create the patient-specific 3-D model (Figure 4).
A total of 184 cases (88.4%) were planned with multimodality, ie, using > 1 image data set (eg, MRI, MRA, CT, CTA, diffusion tensor imaging). Twenty surgeons used the system as a tool to plan their operations preoperatively. Usually, the participating resident prepared the data and the lead neurosurgeon verified or adjusted the segmented objects.
VR surgery planning resulted in a change in surgical strategy compared with planning with conventional 2-D image series in 53 of 208 cases (25.5%). A change in the surgical concept was defined as a significant change in the macrosurgical concept (such as changing the site of the craniotomy from pterional to supraorbital or changing the position of the patient) or a change in the microsurgical concept (changing the surgical route).
We used intraoperative navigation in 61 of the 208 operated cases (29.3%). In 35 of these 61 cases (57.3%), surgeons retrospectively regarded the preoperative VR planning as so sufficient that the use of image-guided navigation would not have been necessary.
The intraoperative scenarios accurately reflected the preoperatively generated virtual models and viewpoints of the Dextroscope (compare illustrative case 1). The analysis of our questionnaire showed good or very good improvement for preoperative spatial understanding (93.3%), preoperative planning of craniotomy size and location (83.1%), intraoperative spatial orientation (74.5%), and intraoperative confidence (74.6%; Table 2).
This 28-year-old woman presented with a progressive palsy of the elevation of the left foot that started about 1 year before surgery. Additionally, she reported hypesthesia of her left leg. Preoperative MRI revealed a central tumor with little contrast enhancement (Figure 5A and 5B). Surgery was planned by creating a multimodal, patient-specific VR model. We used a combination of T1 contrast-enhanced sagittal MRI and a fluorodeoxyglucose PET, an MRA displaying the venous system, a diffusion tensor imaging MRI data set, and a CT scan. Combining these various image modalities allowed the display of not only the tumor localization but also the position of the more active parts of the tumor in relation to the craniotomy, a prominent cortical vein, and the motor cortex (Figure 6A, B). Using the fluorodeoxyglucose PET scans allowed us to determine the active parts of the tumor; these images were then used to define the size of the tumor in the VR patient model.
Close inspection of the virtual patient model demonstrated that the ideal position for the corticotomy should be posterior to the prominent draining vein (Figure 7A and 7B). Surgery was subsequently carried out with the help of intraoperative navigation and resection control by ultrasound. Intraoperatively, the virtual patient model proved to be concordant with the intraoperative situation. At the chosen entry point for the corticotomy, we found no motor answer using cortical mapping with the patient being awake during surgery (Figure 7C). The postoperative MRI showed gross tumor removal (Figure 8). Histological examinations revealed a grade 2 and partly grade 3 astrocytoma. In the postoperative course, the palsy of the left foot was initially more prominent; however, the patient recovered well, and after 3 weeks of rehabilitation. we found only an altered gait.
In a prospective series, we used the system from November 2010 until January 2011 in 33 cases and 32 patients to plan intracranial neurosurgical operations.
All patient data were successfully loaded and displayed on the MD20-3-D system. The hardware and software worked without any failure. The time needed to create the stereoscopic model from the beginning of the data import until the beginning of the planning procedure was about 5 minutes.
Table 3 summarizes the cases examined and the potential benefits and drawbacks identified for each patient.
We did not encounter adverse effects of the autostereoscopic presentation such as motion sickness, nausea, or fatigue. However, we did not administer motion sickness questionnaires. The image quality on the autostereoscopic monitor was comparable to that of a normal computer monitor, the only difference being that the background always had to be black. We saw no apparent differences in image resolution or clarity when comparing the same 3-D image displayed on the autostereoscopic monitor and on the regular 2-D LCD monitor positioned next to it.
Advantages of the Setred System
The evaluation of our questionnaire showed the expectations on which the surgeon used the MD20-3-D. They were the following: easy establishment of the surgical strategy (craniotomy and/or patient positioning; expectations/expectations fulfilled, n = 33/33) and better visualization of pathology and surrounding anatomic structures (expectations/expectations fulfilled, n = 9/8). In 24 cases, the system served as navigation backup solution (expectations fulfilled in 24 cases). In all of these cases, the intraoperative use of a navigation system was intended by the surgeon. In 3 of those 24 cases (cases 9-11), the intended intraoperative use of the navigation system was not possible owing to a mismatch of the navigation system after draping (case 9), because of software malfunction of the navigation system (case 10), or because the navigation system was needed for a different operation taking place at the same time (case 11). However, with the use of screenshots generated during the planning process, including measurements in the cranium and along the skin, the surgery could be carried out as intended (illustrative case 2).
We found that using the autostereoscopic monitor system offered the possibility to preoperatively test different approaches (eg, pterional vs subfrontal approach), especially in aneurysms of the anterior circulation. Moreover, the autostereoscopic monitor system allowed an interactive discussion of the case between several neurosurgeons. In 3 of the 33 cases, VR surgery planning led to a change in surgical strategy as defined above.
Drawbacks of the Setred System
Although the actual autostereoscopic monitor is perfectly suitable for clear and pleasant 3-D viewing, the shortcoming of the system lies in the planning features of the software. First, we missed the display of multimodal data. This feature has been implemented in later software versions; however, our experiences with the multimodality option of the MD20-3-D are limited. In 8 of our 33 cases, the lack of multimodality was criticized. Second, dedicated curvilinear measuring tools that allow the user to measure along the surface of the skin or the skull were missed in 7 of 33 cases. Third, the process of color coding can, in certain cases, be somewhat complicated. In cases when a preset cannot be used, the user has to manipulate the color adjustment table manually. This trial-and-error method is not intuitive, is time-consuming, and was criticized in many cases. Furthermore, the system lacks the essential segmentation tools necessary to create a 3-D model consisting of a variety of subsegmented objects like the cortex, a tumor, ventricles, or cranial nerves. Finally, the user interface of the Setred system to work with 3-D data is limited to manipulations controlled with the computer mouse. Especially when working in the depth of the volumetric data, a 3-D user interface is missing.
Patient 10 suffered from a right-sided hemiparesis and motoric aphasia. The preoperative MRI revealed an intraparenchymal frontal tumor on the left side with inhomogeneous contrast medium uptake (Figure 9A-C). Cancer history was unremarkable. Surgery was planned by use of a sagittal, contrast-enhanced gradient-echo 3-D MRI image series. The same series was used for intraoperative navigation. The DICOM data were transferred to the MD20-3-D and volumetrically reconstructed (Figure 10A and 10B). We then planned the left frontal craniotomy. The position of the craniotomy was clearly recognizable because of the fiducial markers attached to the skin.
Intraoperatively, neuronavigation failed because the USB stick containing the patient data was not recognized by the navigation system. We therefore used a screenshot of the MD20-3-D system showing the planned craniotomy in relation to the position of the fiducials attached to the skin (Figure 10A and 10B). Surgery was uneventful, and the tumor was removed totally. The postoperative CT scan showed tumor removal with a persisting brain edema (Figure 11). Histology revealed a metastasis of a colon carcinoma that was subsequently identified by a colonoscopy.
Over the past years, there has been increasing public interest in the stereoscopic display of images, and even the movie industry has picked up on the idea of adding depth to the big screen. The principles and the technology of 3-D display, however, are not new. Charles Wheatstone15 described the underlying physiological processes as early as 1838. He even described an instrument allowing viewing of pictures stereoscopically and called it a “stereoscope.” In the medical field, there have been only a few reports on using stereoscopic monitor systems, and most of those systems require the use of red-green or polarizing glasses.16,17
The advantages attributed to stereoscopically displayed data are the improvement of subjective image quality, improved separation of an object of interest from its visual surrounding, better depth judgment, and improved surface detection. Hence, the main applications of stereoscopic display techniques are in the fields of radiological diagnosis, preoperative surgical planning, training, and teaching.18
Lately, there have been many reports on the benefits of thorough preoperative surgery planning with VR workstations.19-23 By presenting a stereoscopically perceivable virtual patient model, these systems allow the user to follow and recognize the intrinsic 3-dimensionality of a neurosurgical procedure. Despite this major advantage over conventional surgery planning, these systems have not yet seen a widespread acceptance. According to Davis et al,24 the user acceptance of a new technology is influenced by 2 factors. First and most important, there needs to be a perceived usefulness for the user. Second, the perceived ease of use is a considerable determinant of people’s intention to use new technologies. Thus, successful acceptance of this still novel technology of VR surgery planning means convincing neurosurgeons that there are important benefits and that this technology is easy to use.
We have been working with the Dextroscope for > 6 years and are still using it on a regular basis for surgery planning. To the best of our knowledge, it was the first system allowing 3-D reconstruction and simultaneous multimodal display of several imaging series and their segmented subvolumes, providing a variety of segmentation tools, volume exploration methods, and most important, dedicated surgical planning tools. Furthermore, the Dextroscope introduced a novel kind of user interface that suspends the 3-D data in a virtual workspace, breaks free from the mouse and keyboard interface, and facilitates intuitive interaction in 3-D space. Using the Dextroscope in a series of 208 patients who underwent minimally invasive neurosurgical operations, we found several advantages. Given the depth perception of the stereoscopic display of the Dextroscope, the preoperative work with the virtual patient data resulted in an entirely new understanding of the individual anatomy of each patient. With the reduction in guesswork, the surgical approaches could be tailored to individual anatomic considerations, and hence the situation of “what you see is what you get” could largely be avoided. We conceptualized that a change in surgical strategy after planning in a 3-D environment compared with decisions based on conventional planning would demonstrate the impact of the 3-D planning process. Hence, we asked the surgeons to plan the surgery using regular CT and/or MRI series before using the Dextroscope. We found that in a quarter of cases (25.5%), the surgeons reached a different conclusion concerning the best way to perform surgery after 3-D planning. This confirms our results published in 2008 (23%).13
We found that 88.4% of our Dextroscope cases were built by combining ≥ 2 imaging modalities and that segmentation tools to create subvolumes were applied in more than two-third of the cases. The level of detail in most of the 3-D models might explain the fact that our questionnaires reported good or very good improvement in preoperative spatial understanding in 93%, preoperative planning of craniotomy size and location in 83%, intraoperative spatial orientation in 75%, and intraoperative confidence in 75%. Rosahl and coworkers25 have described similar results and summarized the combined experiences gathered during preoperative planning and recalled during the surgery nicely with the term dèjà vu.
When evaluating the Dextroscope, we found that 1 drawback is the time needed to process the data to build the 3-D patient-specific models; however, we observed a constant decrease of data preparation time over the years: from > 2 hours in the early days to a current average of about 30 minutes or less. Several reasons account for this positive development. First, many of the steps to create 3-D models have been automated; the most important are image coregistration and segmentation methods. Second, as with every other software, the experience resulting from continuous use facilitates workflow and helps maximize the potential benefit of all the software features.
MD20-3-D Planning Station
Driven by the desire to reflect the depth and vividness of the real world on movie screens and televisions, technology of stereoscopic viewing has been developed, although special goggles are still needed. However, the wearing of goggles is regarded as an obstacle to establishing 3-D in consumer electronics (US patent 7843449, Apple Inc, Cupertino, California). Here, autostereoscopic monitors might be a solution in helping 3-D visualization technologies become accepted by users. Although commercially available, autostereoscopic monitor systems are not used much in medicine. Only recently have there been reports on the use of this technology in laboratory settings.26,27
We could not identify articles related to autostereoscopy in a clinical setting. The only article that used patient-specific data was that of Abildgaard and colleagues.12 However, they used data from patients who had already been diagnosed with normal MRAs. In that respect, they used a “laboratory setting,” trying only to evaluate the potential benefits of this technology. Hence, our report is most likely the first on the potential benefits of presenting patient data on an autostereoscopic monitor system. Our findings regarding the preoperative use of an autostereoscopic system are in line with the positive results published by other groups working with 3-D planning stations and with our own experiences with the Dextroscope. Working with the 3-D data on the MD20-3-D was found to be intuitive because it reflected the intraoperative situation in a clear and pleasant work space. Surgery planning was carried out in a rather simple but efficient way. Data preparation was quite straightforward, which of course was also due to the fact that the system, compared with the Dextroscope, does not offer a wide range of image fusion, segmentation, and planning tools. Usually as a first step, skin, tumor, or vessels were visualized in contrasting colors with a color adjustment table. The virtual patient model was subsequently rotated according to the intraoperative position. Finally, various trajectories and magnifications were simulated to understand different viewpoints and the structures along this approach. For vascular cases, particularly aneurysms, we found the system to be especially useful: MRA or CTA can easily be segmented, and the autostereo display is suitable for studying the anatomy of the neck and the parent and branching vessels. The system also allowed a quick understanding of tumor-skin relationships, and anatomic landmarks like the coronal suture or MRI fiducials facilitated this task. This made craniotomy planning quite easy, and the additional intraoperative use of screenshots of the 3-D model was helpful for quickly verifying whether the navigation system indicated the correct position of the craniotomy. We found that the Setred system lacks dedicated tools to measure curvilinear distances, especially along the skin surface. We worked around this shortcoming by adding several straight linear lines. In 1 case of navigation system failure, we could accurately carry out the skin incision and the craniotomy based on the 3-D planning procedure. This is in line with our experiences reported previously and with the results of our Dextroscope series reported in this article that in 57% of the cases navigated intraoperatively after Dextroscope planning, the surgeons stated that the intraoperative use of navigation was not regarded as necessary.23 Keeping in mind that neuronavigation is, in 40% of cases, used only to define the position of the craniotomy,28 that failures of navigation occur in 5% to 13%, and that an error of 1 to 1.5 cm may occur when a craniotomy is measured with traditional techniques,29 the need to have backup solution becomes quite understandable. A precise 3-D planning station is such a backup solution. However, only a randomized, controlled trial comparing neuronavigation and VR surgery planning could definitely clarify this statement. The literature reports several cases of motion sickness and postural instability when VR simulators based on 3-D glasses are used.30 One important finding with the autostereoscopic monitor was that we did not encounter these problems. Akiduki and colleagues30 state that “abnormality of spatial orientation provokes motion sickness leading to autonomic symptoms and produces postural instability.” They showed that an incoherent visual-vestibular conflict induced by VR resulted in motion sickness and postural instability. We can only speculate why this effect was not observed when the MD20-3-D autostereoscopic monitor system was used. Because the technology of the MD20-3-D allows the user to view the 3-D object as a volume within a 90° viewing angle, the above-mentioned visual-vestibular conflict might be avoided.
Comparing the 2 Systems
The Dextroscope provides comprehensive visualization and planning tools, multimodal display, and a 3-D user interface. The Setred system provides a clear volumetric display on an autostereoscopic monitor with the essential tools of color coding and volume exploration. Multimodality display based on accurate autofusion is currently being implemented in the Setred system, which is indeed a key element of comprehensive surgery planning, as several groups have confirmed. Barnett and Nathoo31 described the intraoperative use of multimodal image data sets in their article on the future of the neurosurgical operating room. Tanaka and colleagues32 pointed to potential advantages of MET and PET fusion in planning and neuronavigation as a promising technique. Kockro et al33 described a navigated augmented reality image system with video overlay using multimodal images. Ortler and collleagues34 used multimodality imaging successfully in the surgical management of patients with refractory epilepsy. In fact, the lack of a detailed multimodality and multisegmented display probably explains the lower rate of change in surgical strategy with the MD20-3-D (3 of 33 cases) compared with the Dextroscope (25.5%).
The combination of the clear and convenient autostereoscopic display of Setred and the functions and 3-D interface of the Dextroscope would likely result in a powerful next-generation neurosurgical planning system. On that basis, more refined segmentation tools and methods that may offer some prediction of soft-tissue deformation could add significant value to the planning and simulation process. Ultimately, presurgical 3-D planning should seamlessly lead to intraoperative navigation based on the same 3-D information. Especially in the operating room, putting on and taking off polarizing glasses for the sake of navigation is cumbersome; thus, an autostereoscopic display could be of decisive value for navigating in a virtual 3-D space while operating in a complex 3-D territory.
From our experiences with the 2 systems described in this article, we conclude that planning of neurosurgery with 3-D computer graphical models compared with planning with 2-D image series leads to improved anatomic understanding of the surgical target zone and the corridor toward it, refined surgical strategies, and improved intraoperative confidence. The 3-D visualization technology and spatial user interfaces are providing powerful means for surgeons to plan a specific approach by taking into account its entire 3-dimensionality, thereby improving the understanding of the visible surgical territory and the invisible structures beyond.
Dr Kockro is one of the founders of Volume Interactions Pte Ltd, the company that developed the Dextroscope. Drs Stadie and Kockro have been advising Setred SA, the company that manufactures the MD20-3-D system. They are not receiving any financial compensation for their advisory activity.
We thank Eike Schwandt, MD, of the Department of Neurosurgery, Universitätmedizin Mainz, for supporting this article with his images of the Dextroscope illustrative case.
1. Kockro RA, Serra L, Tseng-Tsai Y, et al.. Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery. 2000;46(1):118–135; discussion 135-137.
2. Batnitzky S, Price HI, Lee KR, et al.. Three-dimensional computer reconstructions of brain lesions from surface contours provided by computed tomography: a prospectus. Neurosurgery. 1982;11(1 pt 1):73–84.
3. Chalif DJ, Dufresne CR, Ransohoff J, McCarthy JA. Three-dimensional computed tomographic reconstructions of intracranial meningiomas. Neurosurgery. 1988;23(5):570–575.
4. Lansky LL, Batnitzky S, Price HI, Cook PN, Dwyer SJ 3rd. Application of three-dimensional computer reconstruction from computerized tomography to intracranial tumors in children. J Neurooncol. 1983;1(4):347–356.
5. Kikinis R, Gleason PL, Moriarty TM, et al.. Computer-assisted interactive three-dimensional planning for neurosurgical procedures. Neurosurgery. 1996;38(4):640–649; discussion 649-651.
6. Oldhafer KJ, Högemann D, Stamm G, Raab R, Peitgen HO, Galanski M. 3-Dimensional (3-D) visualization of the liver for planning extensive liver resections [in German]. Chirurg. 1999;70(3):233–238.
7. Wadley J, Dorward N, Kitchen N, Thomas D. Pre-operative planning and intra-operative guidance in modern neurosurgery: a review of 300 cases. Ann R Coll Surg Engl. 1999;81(4):217–225.
8. Hu XP, Tan KK, Levin DN, et al.. Three-dimensional magnetic resonance images of the brain: application to neurosurgical planning. J Neurosurg. 1990;72(3):433–440.
9. Nakajima S, Atsumi H, Bhalerao AH, et al.. Computer-assisted surgical planning for cerebrovascular neurosurgery. Neurosurgery. 1997;41(2):403–409; discussion 409-410.
10. Gandhe AJ, Hill DL, Studholme C, et al.. Combined and three-dimensional rendered multimodal data for planning cranial base surgery: a prospective evaluation. Neurosurgery. 1994;35(3):463–470; discussion 471.
11. Kockro RA, Serra L, Tsai YT, et al.. Planning of skull base surgery in the virtual workbench: clinical experiences. Stud Health Technol Inform. 1999;62:187–188.
12. Abildgaard A, Witwit AK, Karlsen JS, et al.. An autostereoscopic 3D display can improve visualization of 3D models from intracranial MR angiography. Int J Comput Assist Radiol Surg. 2010;5(5):549–554.
13. Stadie AT, Kockro RA, Reisch R, et al.. Virtual reality system for planning minimally invasive neurosurgery: technical note. J Neurosurg. 2008;108(2):382–394.
14. Serra L, Hern N, Guan CG, et al.. An interface for precise and comfortable 3D work with volumetric medical datasets. Stud Health Technol Inform. 1999;62:328–334.
15. Wheatstone C. Contributions to the physiology of vision: part the first: on some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos Trans R Soc Lond. 1838;128:371–394.
16. Unsgaard G, Ommedal S, Rygh OM, Lindseth F. Operation of arteriovenous malformations assisted by stereoscopic navigation-controlled display of preoperative magnetic resonance angiography and intraoperative ultrasound angiography. Neurosurgery. 2005;56(2 suppl):281–290; discussion 281-290.
17. Mathiesen T, Peredo I, Edner G, et al.. Neuronavigation for arteriovenous malformation surgery by intraoperative three-dimensional ultrasound angiography. Neurosurgery. 2007;60(4 suppl 2):345–350; discussion 350-351.
18. van Beurden M, van Hoey G, Haralambos H, Ijsselsteijn W. Stereoscopic displays in medical domains: a review of perception and performance effects. Proc SPIE. 2009;7240(2009):72400A–72400A-15.
19. Fischer G, Stadie A, Schwandt E, et al.. Minimally invasive superficial temporal artery to middle cerebral artery bypass through a minicraniotomy: benefit of three-dimensional virtual reality planning using magnetic resonance angiography. Neurosurg Focus. 2009;26(5):E20.
20. Wong GK, Zhu CX, Ahuja AT, Poon WS. Stereoscopic virtual reality simulation for microsurgical excision of cerebral arteriovenous malformation: case illustrations. Surg Neurol. 2009;72(1):69–72; discussion 72-73.
21. Wong GK, Zhu CX, Ahuja AT, Poon WS. Craniotomy and clipping of intracranial aneurysm in a stereoscopic virtual reality environment. Neurosurgery. 2007;61(3):564–568; discussion 568-569.
22. Ng I, Hwang PY, Kumar D, Lee CK, Kockro RA, Sitoh YY. Surgical planning for microsurgical excision of cerebral arterio-venous malformations using virtual reality technology. Acta Neurochir (Wien). 2009;151(5):453–463; discussion 463.
23. Stadie AT, Kockro RA, Serra L, et al.. Neurosurgical craniotomy localization using a virtual reality planning system versus intraoperative image-guided navigation. Int J Comput Assist Radiol Surg. 2011;6(0):565–572.
24. Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Management Science. JSTOR. 1989;35(8):982–1003.
25. Rosahl SK, Hubbe U, Strahl S, et al.. Déjà vue in the operating room: three-dimensional volumetric imaging. Int Congr Ser. 2005;1281:788–792.
26. Liao H, Inomata T, Sakuma I, Dohi T. 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng. 2010;57(6):1476–1486.
27. Liao H, Ishihara H, Tran HH, Masamune K, Sakuma I, Dohi T. Precision-guided surgical navigation system using laser guidance and 3D autostereoscopic image overlay. Comput Med Imaging Graph. 2010;34(1):46–54.
28. Wagner W, Gaab MR, Schroeder HW, Tschiltschke W. Cranial neuronavigation in neurosurgery: assessment of usefulness in relation to type and site of pathology in 284 patients. Minim Invasive Neurosurg. 2000;43(3):124–131.
29. Spivak CJ, Pirouzmand F. Comparison of the reliability of brain lesion localization when using traditional and stereotactic image-guided techniques: a prospective study. J Neurosurg. 2005;103(3):424–427.
30. Akiduki H, Nishiike S, Watanabe H, Matsuoka K, Kubo T, Takeda N. Visual-vestibular conflict induced by virtual reality in humans. Neurosci Lett. 2003;340(3):197–200.
31. Barnett GH, Nathoo N. The modern brain tumor operating room: from standard essentials to current state-of-the-art. J Neurooncol. 2004;69(1-3):25–33.
32. Tanaka Y, Nariai T, Momose T, et al.. Glioma surgery using a multimodal navigation system with integrated metabolic images. J Neurosurg. 2009;110(1):163–172.
33. Kockro RA, Tsai YT, Ng I, et al.. Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery. 2009;65(4):795–807; discussion 807-808.
34. Ortler M, Trinka E, Dobesberger J, et al.. Integration of multimodality imaging and surgical navigation in the management of patients with refractory epilepsy: a pilot study using a new minimally invasive reference and head-fixation system. Acta Neurochir (Wien). 2010;152(2):365–378.
Keywords:Copyright © by the Congress of Neurological Surgeons
Autostereoscopic display; Autostereoscopy; Stereoscopy; Surgery Planning; 3-D visualization; Virtual reality