Surgical Simulation of Cerebrovascular Disease With Multimodal Fusion 3-Dimensional Computer Graphics

Saito, Nobuhito MD, PhD; Kin, Taichi MD, PhD; Oyama, Hiroshi MD, PhD; Yoshino, Masanori MD; Nakagawa, Daichi MD; Shojima, Masaaki MD, PhD; Imai, Hideaki MD, PhD; Nakatomi, Hirofumi MD, PhD

doi: 10.1227/01.neu.0000430312.71326.6d

Although recent advancements in medical imaging technology have allowed detailed preoperative examinations, neurosurgeons still have to interpret large amounts of medical imaging data. In various modalities such as computed tomography (CT), magnetic resonance imaging (MRI), and angiography, there are multiple sequences and 3-dimensional (3-D) images, and it is not uncommon for there to be several hundred to several thousand section images per case.1 Clinicians have to interpret each of these multimodalities/sequences individually and consolidate this information in their heads to form a 3-D image that can be used in preoperative planning. From the perspectives of accuracy, reproducibility, and sharing information with other people, it is hard to ensure sufficient precision.2 Furthermore, the spatial resolution of the 3-D images used in today’s clinical settings is inferior to that in 2-dimensional (2-D) imaging because the processing methods are limited. Consequently, ascertaining detailed findings from 3-D images alone is unsatisfactory; clinicians must additionally interpret 2-D section images of the same site.3 In this report, we describe the fusion of all image data required for preoperative examination and the construction of 3-D computer graphics (3-DCG) with a high spatial resolution using our own image processing technique. We then apply this to surgical strategies in cerebral vascular disease and report our experience and the usefulness of the technique.

Back to Top | Article Outline


Patient Population

Subjects included patients with cerebral vascular disease who underwent preoperative examination using 3-DCG with the proposed fusion technique at the University of Tokyo, Department of Neurosurgery. The internal review board of University of Tokyo Hospital approved the study protocol, and written informed consent was obtained from all patients before participation in this study.

Back to Top | Article Outline

Image Processing

The image processing technique was as per the technique reported earlier and is outlined briefly below.1-5 CT, magnetic resonance imaging, and 3-D rotational angiography (3-DRA) images were used for 3-D reconstruction. Preoperatively recorded images were output in the DICOM (digital imaging communication in medicine) format and imported into the image processing software Avizo 6.3 (Visualization Science Group, Bordeaux, France). For registration of different modalities, automatic registration was performed with the normalized mutual information method using our original initialization method that we have devised.1 Images were presented using a hybrid rendering method in which the surface-rendering method was combined with the volume-rendering method.2 For surface rendering, we used the method of setting multiple modalities and multiple thresholds for a target tissue (multimodal individualizing tissue threshold method) to obtain high spatial resolution.1 The 3-D images (fusion 3-DCG) completed on the workstation were used to carry out the interactive virtual reality operative simulation.

Back to Top | Article Outline


The proposed 3-DCG fusion technique facilitates the rendering of 1 fusion 3-DCG with a high spatial resolution. Errors in the simulation did not pose any clinical problems, and there were no diagnostic errors caused by fusion 3-DCG. In the surgical simulation, fusing each modality made it possible to comprehensively understand the pathological condition in 3-D. We describe here typical cases in which fusion 3-DCG with a high spatial resolution was beneficial in terms of surgical strategy because many findings were observed in fusion 3-DCG that could not be found in nonfused images.

Back to Top | Article Outline

Case 1: Aneurysm of the Left Posterior Cerebral Artery

Various virtual surgical sites were examined preoperatively with the proposed 3-DCG fusion technique. In this case, the 3-D reconstruction was created using a 3-DRA of the 3-D model of the vessel, fast imaging employing steady-state acquisition of the 3-D model of the cortex, and CT of the 3-D model of the skull. In this case, the posterior transpetrosal approach (Figure 1A) or subtemporal approach (Figure 1B) was examined as a virtual surgical site, and the aneurysm was actually reached by a subtemporal approach. Fusion 3-DCG with the proposed method makes it easy to understand the lesion and the anatomy around the lesion in detail and in 3-D, even in surgical sites that are relatively uncommon such as in this case (Figure 1C and 1D).

Back to Top | Article Outline

Case 2: Left Paraclinoid Aneurysm

It is possible to manipulate various surgical simulations with fusion 3-DCG. This is a case of double bilobular paraclinoid aneurysms (Figure 2A). The anterior clinoid process was virtually removed (Figure 2B); the optic nerve and flow of the ophthalmic artery were confirmed through the semitransparency of the skull (Figure 2C); and virtual reality simulation of neck clipping with a virtual clip was conducted (Figure 2D).

Back to Top | Article Outline

Case 3: Aneurysm of the Distal Anterior Cerebral Artery

The characteristics of the aneurysmal wall are often obtained from multiple modalities/sequences. This is a case of an aneurysm associated with thrombus, hematoma, and calcification (Figure 3A-3C). With just time-of-flight MR angiography, only the part of the aneurysm with blood flow can be visualized in 3-D (Figure 3D), whereas fusion 3-DCG allows the thrombus, hematoma, and calcification parts of the aneurysm to be displayed as 1 CG (Figure 3E), thus allowing the correct shape of the aneurysm to be found (Figure 3F).

Back to Top | Article Outline

Case 4: Arteriovenous Malformation

In arteriovenous malformations, identifying the feeder and drainer is an extremely important surgical strategy. No matter how detailed an interpretation is made of medical images of vascular information such as in 2-D digital subtraction angiography, it is difficult to understand the positional relationship of cerebral parenchyma and blood vessels. Fusion 3-DCG allows a detailed understanding of the positional relationship between the arteriovenous malformation and the cortical sulcus and gyrus (Figure 4A), and even with blood vessels that can be only partially identified from the brain surface, it is easy to distinguish the feeder from the drainer. Fusion 3-DCG is very useful when identifying the feeder and drainer in actual surgery (Figure 4B).

Back to Top | Article Outline

Case 5: Brainstem Cavernous Malformation

Brainstem cavernous malformations associated with developmental venous anomalies often cannot be identified in preoperative image findings (Figure 5A). In fusion 3-DCG with high-resolution imaging through 3-DRA, developmental venous anomalies that could not be identified by 2-D digital subtraction angiography were successfully rendered in detail (Figure 5B). In this case, the most accessible point was identified by use of the 2-point method6 on fusion 3-DCG, and the virtual surgical view showing retraction of the cerebellum (Figure 5C) was very similar to actual surgical findings (Figure 5D).

Back to Top | Article Outline

Case 6. Left Temporal Pial Arteriovenous Fistula

Fusion 3-DCG using the proposed method was possible with the fusion of functional images and fluid analysis image data.7 This is a case of pial arteriovenous fistula with the shunting point on the brain surface; in this case, diagnosis was very difficult on 2-D digital subtraction angiography (Figure 6A). Localized in the left frontal lobe (Figure 6B) and using fusion 3-DCG to ascertain the positional relationship between the lesion with the corticospinal tract, arcuate fiber and optic radiation was useful in surgical examinations (Figure 6C). Blood flow analysis data using the mesh from the finite-element method7,8 were fused to create CG, which helped to identify the shunting point (Figure 6D).

Back to Top | Article Outline

Case 7: Spinal Dural Arteriovenous Fistula

Spinal angiography is the gold standard for the diagnosis of spinal dural arteriovenous fistula, in which it is often difficult to identify the shunting point and to ascertain the 3-D positional relationship with the lesion (Figure 7A). The positional relationship of the arteriovenous fistula, bone, and dura mater is a very important factor in surgical planning. In this case, fusion 3-DCG images were created that reconstructed a 3-D model of the bone from plain CT, a 3-D model of the dura mater from the postmyelography CT (Figure 7B), and a 3-D model of the vessels from spinal 3-DRA (Figure 7C).4 In this fusion 3-DCG, the abnormal vessel and the dura mater penetration site were easily identified, and the virtual surgical view (Figure 7D) of the virtual bone removal was consistent with actual surgical findings (Figure 7E).

Back to Top | Article Outline


We reported that surgical simulations of cerebrovascular disease with high-resolution multimodal fusion 3-dimensional images are useful.

Although the normalized mutual information method is widely used in medical image registration methods,9,10 there is a problem in that, unless the proper initial values are set, correct interpretation cannot be obtained, and in registration of > 2 modalities, there are often difficulties.10 In the technique proposed here, multimodal registration was made possible by focusing on the luminance range from the initial voxel size, the distribution, and the image range.1

Visualization of medical 3-D imaging can be broadly divided into the surface-rendering method and the volume-rendering method. The former divides the object surface into micropolygons to render an image, and the latter displays the entire selected threshold for an image as a 3-D image. When complex 3-D images that fuse multiple modalities are visualized, there are limitations to both rendering methods. In our proposed method, we used a hybrid rendering method in which we used mainly the surface-rendering method and displayed 3-DCG simultaneously as required using the volume-rendering method. This way, we succeeded in visually simplifying highly complicated 3-D information.2

In the surface-rendering method, as a general rule, 3-D images are created by selecting 1 threshold pair, although in the data of 1 image the signal intensity differs even with the same tissue, and the threshold with little noise is generally selected even if a certain degree of spatial resolution must be sacrificed. In our proposed method, we divided multimodal images for 1 target tissue into random regions of interest, and by setting the threshold so that each region of interest had a high spatial resolution and low noise, we were able to reconstruct 3-D images with a high spatial resolution.3

Problems with this method include the need for a few hours to process images and the fact that the accuracy verification method for registration and manual segmentation is not completely established. Moreover, the original objective of multimodal fusion 3-DCG is to improve patient prognosis and surgical outcomes. In this respect, further cases should be gathered and examined.

For related video content, please access the Supplemental Digital Content:

Back to Top | Article Outline


The internal review board at University of Tokyo Hospital approved the study protocol, and written informed consent was obtained from all subjects. The authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.

Back to Top | Article Outline


1. Kin T, Nakatomi H, Shojima M, et al.. A new strategic neurosurgical planning tool for brainstem cavernous malformations using interactive computer graphics with multimodal fusion images. J Neurosurg. 2012;117(1):78–88.
2. Kin T, Shin M, Oyama H, et al.. Impact of multiorgan fusion imaging and interactive 3-dimensional visualization for intraventricular neuroendoscopic surgery. Neurosurgery. 2011;69(1 suppl operative):ons40–ons48; discussion ons48.
3. Yoshino M, Kin T, Shojima M, Nakatomi H, Oyama H, Saito N. A high-resolution method with increased matrix size can characterize small arteries around a giant aneurysm in three dimensions. Br J Neurosurg. 2012;26(6):927–928.
4. Takai K, Kin T, Oyama H, et al.. The use of 3D computer graphics in the diagnosis and treatment of spinal vascular malformations. J Neurosurg Spine. 2011;15(6):654–659.
5. Kin T, Oyama H, Kamada K, Aoki S, Ohtomo K, Saito N. Prediction of surgical view of neurovascular decompression using interactive computer graphics. Neurosurgery. 2009;65(1):121–128; discussion 128-129.
6. Brown AP, Thompson BG, Spetzler RF. The two-point method: evaluating brain stem lesions. BNI Q. 1996;12(1):20–24.
7. Shojima M, Nemoto S, Morita A, Oshima M, Watanabe E, Saito N. Role of shear stress in the blister formation of cerebral aneurysms. Neurosurgery. 2010;67(5):1268–1274; discussion 1274-1275.
8. Shojima M, Oshima M, Takagi K, et al.. Magnitude and role of wall shear stress on cerebral aneurysm: computational fluid dynamic study of 20 middle cerebral artery aneurysms. Stroke. 2004;35(11):2500–2505.
9. Estévez PA, Tesmer M, Perez CA, Zurada JM. Normalized mutual information feature selection. IEEE Trans Neural Netw. 2009;20(2):189–201.
10. Pluim JP, Maintz JB, Viergever MA. Mutual-information-based registration of medical images: a survey. IEEE Trans Med Imaging. 2003;22(8):986–1004.
Copyright © by the Congress of Neurological Surgeons