Mitha, Alim P. MD, SM*; Almekhlafi, Mohammed A. MD*,‡; Janjua, Major Jameel J. BEng, SM, CD§; Albuquerque, Felipe C. MD¶; McDougall, Cameron G. MD¶
Compared with other neurosurgical disciplines, endovascular procedures have the greatest dependence on imaging. The precision of interventionalists in performing various complex maneuvers is heavily dependent on their perception of and interaction with an imaging interface as opposed to direct visualization of an open surgical field. Although the gradual refinement of open neurosurgical procedures closely parallels the progress of imaging technology, its place in the advancement of endovascular techniques has not yet been realized. Instead, devices such as modern-day coils, stents, and embolic agents seem to dictate the success of endovascular procedures.
The dependence of endovascular procedures on imaging, however, makes it ideally suited for embracing one of the most significant advancements of our time, ie, virtual reality (VR), in which the real world is replaced by a simulated one. The rationale for this is simple. Multiple studies in task performance have shown that the most efficient transfer of knowledge and skill is achieved when the similarity between the task and the task environment is maximized. Therefore, the ability to create a computerized 2-dimentional or 3-dimensional (3-D) simulation of intravascular procedures either in an artificial scenario or in real time within a living system can allow better understanding of the vascular anatomy, which can be exploited in the training of residents and fellows or while practicing or performing endovascular procedures. These image-based advancements may eventually help to increase the safety and efficacy of endovascular procedures.
Here, the current paradigm of endovascular training and its inherent limitations are described. One author (J.J.J.) is a CF-18 pilot and fighter weapons instructor for the Canadian Air Force. He has extensive experience with simulators as both a trainee and an instructor. Two authors (A.P.M. and C.G.M.) are also involved in aviation as private pilots. We use our collective experience to draw parallels between the benefits of simulation in fighter pilot training and endovascular training. We also review emerging simulation technologies, including VR and augmented reality (AR), in the context of endovascular neurosurgical procedures.
Difficulties in Endovascular Training
Endovascular training currently consists of a 24- to 48-month period based heavily on clinical experience. Fellows typically begin their term by learning the principles of femoral access and diagnostic angiography techniques using conventional catheters. Common fluoroscopic views for imaging particular vascular territories are also emphasized early in their training.1 As trainees demonstrate competence with the simpler procedures, they are allowed to tackle more difficult aortic arch configurations and more technically challenging catheter manipulations. As further competency is demonstrated, the trainee is given progressively increasing responsibility with interventional cases such as aneurysm coiling, embolization, and stenting procedures.
The difficulty of training in the endovascular environment is that the haptic skills and sense of tactile recognition required are generally not developed intuitively.2 That is, the learning occurs by doing and depends on a cultivated sense of judgment. Early on, the process is characterized more by trial and error compared with open surgical procedures, in which directly visualizing the anatomy being manipulated helps give important feedback to the operator. Furthermore, there is only one set of controls. The supervisor can either watch how the trainee is physically manipulating the catheter or see the repercussions of those manipulations on the display, but he or she cannot do both simultaneously.
This process is quite different from how other technical procedures are taught. In aviation, for instance, the instructor sits beside or behind the student and has immediate access to a second set of controls. Initially, the instructor performs a maneuver while the student can feel how the controls are manipulated. Later, the student performs the same maneuver, and the instructor is able to see what the student sees and feel how the controls are being manipulated. Importantly, if the student begins to manipulate the controls in a way that the instructor feels is unsafe, the instructor can immediately correct the error as it occurs and before any significant detrimental consequence.
Unlike aviation and as a result of both the absence of a formal training phase in endovascular skill development and the absence of a second set of controls, the luxury of a clinical instructor being able to prevent a mistake is not always possible. In contrast to teaching a student pilot how to fly, the endovascular supervisor is largely limited to reacting to mistakes after they happen.3 This often comes at a cost because most trainees have experienced any of a number of preventable complications that can be attributed to inexperience such as thromboembolic events, vessel spasm, and dissections.4 There is no question that endovascular trainees learn by doing, but in this age of technology, one should be able to prevent adverse events, to curtail the consequences of an error, and to develop important motor skills before performing the actual procedure.5
Simulation is one method by which one can avert the consequences of a mistake during training. Rehearsal with simulators is a well-established practice in the military and the commercial aviation industry to test various maneuvers or procedures before real-time application.6 In fact, these measures established their roots in aviation, with the realization that safety is highly dependent on reducing the impact of human error.7
Simulation in Fighter Pilot Training
Simulation is used heavily to train fighter pilots. The extensive use of simulation in pilot training is due largely to the complex and expensive nature of flying high-performance military aircraft. Fighter aircrafts cost tens of millions of dollars and have exceptionally high operating costs per hour. In addition, errors in operation or execution of a mission can easily be catastrophic and potentially fatal. The financial costs of such accidents and the safety of crew and passengers make an exceptionally good case for finding ways to mitigate these risks.
Learning to operate a fighter aircraft safely and effectively takes time. There is no single greater factor in determining competency than pilot experience. That is to say, if a pilot has seen a particular event before, whether that event is part of a simulation or during actual operation of the aircraft, then the pilot will be much better prepared to correctly handle the situation again. This is especially true for emergency situations or defensive actions on being engaged by a hostile weapons system, which are much less commonly experienced than routine aircraft operation.
The syllabus to train fighter pilots on the CF-18 in Canada relies on the use of flight simulators as an integral part of the training. The generic concept is that missions or scenarios can be conducted first in a high-fidelity, VR flight simulator with an instructor pilot watching in a separate room. Modern flight simulators allow instructors to monitor all facets of the mission, including additional multiple perspectives and metrics that are not available to the trainee. All of this information is recorded to enable a detailed and extensive postmission debriefing. The instructor and trainee can communicate in real time during the session in case the instructor has immediate feedback. In addition, a replay of the mission in the debriefing will allow the instructor to critique the student’s performance and subsequent results very closely, using cause-and-effect relationships, to better facilitate the overall training goals.
After a simulated mission, the student pilot will practice in a real aircraft the same skills that were learned in the simulator. As described previously, the instructor pilot will normally be in the same aircraft in the backseat, with his or her own set of controls, during the introduction of a critical or complicated airborne task (Figure 1). This allows the instructor to demonstrate the correct procedure before the student tries it in a real aircraft for the first time and to immediately take control of the aircraft in case an unsafe situation develops.
It is important to note that the use of simulators does not end once a student has learned a new skill. Flight simulators are used heavily to master skills and to mitigate the fading of skill in areas that are not often practiced. A fighter pilot who has exceeded his or her flying currency limit (a pilot will lose “currency” after not having flown the actual aircraft for a prescribed number of weeks or months) will have to conduct a simulator mission under supervision to ensure that the pilot still possess the skills required to operate safely in this complex environment.
Training fighter pilots to assess the environment around them, analyze, and then act decisively and correctly in life-threatening scenarios with simulators has tremendous benefit. The benefit of learning decision making under pressure is relatively obvious. Additionally, however, there is the benefit of teaching trainees to instinctively perform the complex mechanical skills associated with flying a high-performance aircraft. Like endovascular training, the motor skills required as part of the tasks of fighter pilots are not always intuitive. Furthermore, they are to be conducted in an environment with many additional stressors, including g-forces, a small cockpit, temperature extremes, and weather factors. In addition, there are mental stresses associated with conducting an operational mission in which success and failure can equate to life and death for the pilot.
The use of simulators in aviation also allows instructors to initially remove external stresses that can further complicate the mission and instead focus on teaching the trainee how to execute a task correctly. Once the foundation is built and the student has demonstrated the correct mechanical skills to the desired standard, the instructor can begin to add other environmental factors to the simulations so that they become more representative of the reality in which these student fighter pilots will eventually operate. VR flight simulators offer a variable environment in which the instructors can ensure that motor memory is established as students conduct both routine and complex tasks in the cockpit and that these motor skills will always provide a basis for sound performance when the real world offers something unforeseen.
Simulation in Endovascular Training
Like aviation, the field of endovascular neurosurgery is an ideal target for simulation. The decision-making skills required for different patient and technical scenarios, as well as the motor skills required for wire manipulation, catheter navigation, and the use of various devices, can be acquired by training within a simulation interface as opposed to during patient procedures.8 With this type of preparation, the natural learning curve can be at least partially limited to scenarios without direct patient consequence.
However, there are more practical reasons for justifying simulation in endovascular training that may perhaps differ from aviation. With the increasing emphasis on less invasive surgery and the increasing demand for endovascular procedures, traditional endovascular training, which was previously limited to postresidency education, is facing new challenges in terms of higher trainee volumes. The Accreditation Council of Graduate Medical Education 2009/2010 National Data Report for Neurological Surgery showed that half of neurosurgical residents reach the senior clinical year without performing a single arteriography procedure.9 As a consequence, required rotations in endovascular neurosurgery have been added to many neurosurgical residency programs. Furthermore, trainees come from various disciplines, including neurosurgery, neurology, and radiology, with varying backgrounds in technical procedures. This increased volume of training, however, can potentially increase risk and radiation exposure for the patient, as well as increasing the length of the procedure. Ethical and practical concerns, including materials costs, infrastructure, and personnel requirements, tend to limit the use of animal models or cadavers for this purpose.10 Current endovascular simulators include either pump-driven systems or computerized, VR systems and are detailed further below.
Pump-Driven Flow Models
Pump-driven flow models are a simple, available, and affordable technique for improving procedural skills. They generally involve an external pump driving flow through a set of Silastic tubes or a transparent silicone block phantom that mimics the cerebral circulation (Figure 2). They have the advantage of direct visualization as the wire, catheter, or devices are maneuvered into target locations. The operator gains real-time tactile and visual feedback while working with the catheters and devices and learns how it reacts to the step-by-step manipulation. Furthermore, several companies can create patient-specific anatomic conformations of the tubing based on imaging data. However, there are several significant drawbacks to this modality. The anatomy is generally fixed, and haptic feedback from catheter manipulations within the model is unrealistic. In addition, local and systemic physiological parameters are lacking.11
Computer VR Training
High-fidelity, computerized VR simulation systems overcome many of the limitations of traditional training methods and pump-driven flow models by providing high-quality, realistic, and interactive training in a controlled, risk-free environment.12 Although well established in other highly skilled disciplines such as aviation, the use of VR technology in physician training was only first proposed in 1993.13 Since then, VR simulators have become commercially available, and VR-based training has made steady steps toward establishing its use in many disciplines of medicine.
Consistent with this ideology is the ability of VR simulation to portray a number of potential procedure-related complications, similar to the emergency scenarios of flight training. As any interventionalist will attest, endovascular procedures that are largely considered routine can often pose unexpected difficulties. Tortuous anatomy, propensity for thrombus formation, and vessel spasm are only a few of many factors that can be unexpected and introduce higher levels of complexity to what would otherwise have been a straightforward procedure. Simulated exercises can improve the ability of trainees to detect and react to adverse events that they may encounter during their careers. Criticism of simulated complications, however, includes the viewpoint that these scenarios lack the consequences of clinically adverse events and may fail to emphasize the sense of caution necessary in actual procedures. In one study, simulated complications were actually observed as having very little real-world value for the trainee.14
Nevertheless, interest in VR technology has been further increased by demands from the Accreditation Council of Graduate Medical Education to incorporate resident proficiency-based assessments. This has led programs such as general surgery and peripheral endovascular radiology to adapt VR technology to meet these requirements. The use of simulation in endovascular neurosurgery training, however, trails behind these other disciplines.15 The alternative is more intensive clinical training, which also comes at a financial cost to the healthcare system; use of the operating room for resident teaching is estimated at roughly $50 000 per surgery resident over 4 years. This cost is attributed primarily to prolonged operative times and decreased efficiency.16 Because many endovascular fellowship programs require a minimum of 100 angiograms from aspiring residents, VR simulators may be the ideal solution to at least partially meet this need.
A number of initial studies have shown the positive impact of VR simulators on the skill, knowledge, and interest of neurosurgery residents in endovascular techniques. In one study, 7 neurosurgery residents underwent a 2-day didactic and hands-on training course.15 The study identified a significant reduction in the time required to complete a 4-vessel angiogram and in the total fluoroscopy times. Such output metrics recorded by the simulator provide the means for objective assessment of improved performance.12 Furthermore, residents who had the opportunity to perform simulated carotid stenting had significantly greater interest in further pursuing endovascular surgery compared with those who did not.17
Although novices appear to attain the greatest benefit from these techniques, fellows and experienced interventionalists have also been shown to improve with these simulators.18,19 There was significant improvement in the procedural skills and shorter fluoroscopy times in the final of 5 attempts in one study using endovascular neurosurgery fellows.20 Ten residents participating in the same study were reported to approach the efficiency of fellows by their third and fourth attempts. This ability to differentiate the performance of trainees at various skill levels is another example of the construct validity of VR simulators. Such findings led to the Food and Drug Administration’s acceptance of a proposal in 2004 permitting the use of a carotid angioplasty and stenting simulator in the training of inexperienced interventionalists for qualification as a treating physician in the Carotid Artery Stenting With Emboli Protection Surveillance trial.21 Similar to aviation, experienced interventionalists can also benefit from simulators by being able to practice specific endovascular skill sets that are not often used.
The applications of VR simulators have expanded from training and assessment of trainees to real-life applications. Interventionalists are now able to import patient-specific imaging data from x-ray, computed tomography (CT), and magnetic resonance (MR) imaging into the simulator for rehearsal and procedural planning.22 This was designed to advance VR simulation from focusing on skill acquisition to addressing patient-specific procedural challenges. This approach can also help in the selection of the optimal angiographic projections and of catheters, stents, and other devices optimal for a patient’s particular anatomy. The application has also been shown to provide residents with better physical and mental readiness and higher perceived control of the procedure.23 VR simulation with patient-specific information also carries the promise of anticipating and reducing real-life complications and optimizing patient safety. Such simulators are still prototypes, however, and their difficulty of use and cost are currently limiting availability.24 In addition, further studies are needed to demonstrate their accuracy in predicting vessel response to interventions such as the propensity for dissection and change in configuration after stent placement.25
Several computer-based VR training simulators are available, including the Procedicus Vascular Intervention System Training simulator (Mentice AB, Gothenburg, Sweden) and the ANGIO Mentor (Simbionix, Cleveland, Ohio; Figure 3). They have realistic interfaces and a selection of different scenarios, and more recent versions have the ability to input patient-specific information. They can also mimic several different complication scenarios, including thrombus formation, dissections, vessel spasm, and aneurysmal hemorrhage during coiling (Figure 4). In addition to training of technical skills, the input of medical management parameters such as the administration of antiplatelet agents by the trainee is possible with these simulators. Various levels of difficulty for the scenario, eg, easy to more challenging aortic arch configurations, can be chosen. As with pump-driven flow models, realistic haptic feedback of these systems is currently lacking, but continuous improvements are being made to generate as representative a scenario as possible.
Despite the advantages of VR simulation, a number of limitations should be acknowledged. An important limitation is that the transferability of skills acquired in VR simulations to actual procedures has not yet been demonstrated in large-scale studies. The high costs of the available simulators limit their widespread availability, which is addressed only partly by the organization of regional meetings for residents and fellows from different training programs to practice on VR simulators.26 In addition, important components of clinical angiography that are not currently reproduced in the VR setting include femoral access issues and the continuous flushing of catheters to avoid thromboembolic complications.20
AR is the live view of a physical environment in which the elements have been amplified, enhanced, or supplemented by computer-generated input such as video and graphics. It is characterized by being registered in 3-D, being interactive in real time, and combining the real-world environment with digital images.27,28
In aviation, an example of AR is the heads-up display and helmet-mounted cueing system, which allow own aircraft performance and target data information to be projected as an overlay for pilots. This overlay is projected onto a glass pane in the canopy directly in front of the pilot’s seat or onto the pilot’s helmet-mounted visor, respectively, so that the pilot can see this information while observing the live physical environment in which the aircraft is operating. Another example of AR in aviation is night vision (Figure 5). Night vision drastically enhances what one sees in low- or no-light conditions. It allows an image to be seen by amplifying even the slightest amount of ambient light to give a useable picture.
These examples of AR in fighter aircraft help the pilot assimilate important information without having to direct attention away from the environment in which the mission is taking place. Having awareness of the world around them supplemented by system information makes the pilots more likely to perform their mission safely and effectively. For example, visual information from heads-up display and helmet-mounted cueing system can be used to correctly identify friend from foe by overlaying different visual cues over friendly and enemy aircraft. This information makes the pilot much less likely to use weapons against the incorrect target through the use of visual cues that vastly amplify the difference between valid targets and friendly air or ground units. Similar technology can be very useful for endovascular cases; eg, the path to a target lesion can be highlighted in one color and the vessels to be avoided can be highlighted in another color.
For AR to occur in endovascular cases, there has to be precise integration of previously collected patient data and real-time anatomic information or information collected from different sources integrated and enhanced on the same image. The advantages of integrating real and computerized scenes have been demonstrated in terms of reducing the likelihood of error and improving both recall and attention control during training and real-world performance.29,30 These positive effects are likely related to the complementary relationship between AR and human cognitive processing.31,32
A recent technological development with imaging software has been a 3-D road mapping technique, which is generated by superimposing a 3-D vessel reconstruction (acquired from a rotational angiography data set) on live 2-dimensional fluoroscopy of the patient (Figure 6).33,34 Alignment between these data sets is not dependent on the registration of image content. Instead, the system software determines alignment from the position of the x-ray source and detectors during acquisition. Importantly, this allows image alignment to be maintained on movement of the C arm or when the source image distance is changed.35 It is useful for determining guidewire/catheter location in relation to the blood vessel and aids in navigation without the need for additional contrast. It also helps to prevent and detect wire perforation or vessel dissection. The main limitation of this technique is that the 3-D reconstruction is limited to the vessel reconstruction at the time the 3-D image was acquired and does not contain real-time information.36
Other AR techniques are being developed but are not yet being readily used for endovascular procedures. These include CT fluoroscopy, which can allow vascular interventions within a CT scanner and is made possible by flat-panel detection, continuous gantry rotation, and rapid data collection and processing.37 Limitations, however, include significant radiation doses to the patient and the operator.38 MR fluoroscopy is also possible, offering high temporal and spatial resolution for endovascular interventions.39 Open-bore magnets can allow easier operation by the interventionalist, but they are currently limited by their low field strength.39 Further development of MR fluoroscopy also requires the improvement of intravascular devices optimized for visualization with MR as opposed to current versions, which are designed for visibility with x-ray fluoroscopy. Three-dimensional computerized image overlay can highlight structures relevant to the intervention. Three-dimensional multimodality road mapping is also being developed and consists of fusing preoperative CT or MR with live x-ray fluoroscopy. This allows real-time visualization of endovascular devices in the context of the patient’s 3-D vascular morphology and its soft-tissue context.40
As in aviation, VR and AR have important applications in the field of endovascular neurosurgery. Practical deficiencies in the training of residents and fellows in endovascular techniques can be circumvented by providing a risk-free and highly realistic simulated training environment. As VR technology becomes more accessible, interventionalists will also capitalize on the opportunity to rehearse difficult procedures using patient-specific information before the actual intervention. Similarly, as the technology of AR develops further, live imaging supplemented by computerized enhancements may eventually provide interventionalists with additional tools resulting in improved safety for both the patient and practitioner.
The authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
1. Qureshi AI, Abou-Chebl A, Jovin TG. Qualification requirements for performing neurointerventional procedures: a report of the Practice Guidelines Committee of the American Society of Neuroimaging and the Society of Vascular and Interventional Neurology. J Neuroimaging. 2008;18(4):433–447.
2. Lamata P, Ali W, Cano A, et al.. Augmented reality for minimally invasive surgery: overview and some recent advances. In: Maad S, ed. Augmented Reality. New York, NY: InTech, 2010:230.
3. Perezgonzalez JD, LEE SY. New technologies for the student pilot. Contemporary issues in aviation education and research. Aviation Education and Research Proceedings. 2009:10–11.
4. Earnshaw JJ, Wyatt MG; Joint Vascular Research Group. Complications in vascular and endovascular surgery: how to avoid them and how to get out of trouble. Surg Pract. 2012;16(3):124.
5. Rousseau H, Chabbert V, Maracher MA, et al.. The importance of imaging assessment before endovascular repair of thoracic aorta. Eur J Vasc Endovasc Surg. 2009;38(4):408–421.
6. Krebs WK, McCarley JS, Bryant EV. Effects of mission rehearsal simulation on air-to-ground target acquisition. Hum Factors. 1999;41(4):553–558.
7. Shappell S, Detwiler C, Holcomb K, Hackworth C, Boquet A, Wiegmann DA. Human error and commercial aviation accidents: an analysis using the human factors analysis and classification system. Human Factors: The Journal of the Human Factors and Ergonomics Society. 2007;49(2):227–242.
8. Tedesco MM, Pak JJ, Harris EJ Jr, Krummel TM, Dalman RL, Lee JT. Simulation-based endovascular skills assessment: the future of credentialing? J Vasc Surg. 2008;47(5):1008–1011; discussion 1014.
9. King EG, Weily HS, Genton E, Ashbaugh DG. Consumption coagulopathy in the canine oleic acid model of fat embolism. Surgery. 1971;69(4):533–541.
10. Gallagher AG, Cates CU. Approval of virtual reality training for carotid stenting: what this means for procedural-based medicine. JAMA. 2004;292(24):3024–3026.
11. Gould DA, Kessel DO, Healey AE, Johnson SJ, Lewandowski WE. Simulators in catheter-based interventional radiology: training or computer games? Clin Radiol. 2006;61(7):556–561.
12. Tsang JS, Naughton PA, Leong S, Hill AD, Kelly CJ, Leahy AL. Virtual reality simulation in endovascular surgical training. Surgeon. 2008;6(4):214–220.
13. Satava RM. Virtual reality surgical simulator: the first steps. Surg Endosc. 1993;7(3):203–205.
14. Berry M, Lystig T, Beard J, Klingestierna H, Reznick R, Lönn L. Porcine transfer study: virtual reality simulator training compared with porcine training in endovascular novices. Cardiovasc Intervent Radiol. 2007;30(3):455–461.
15. Fargen KM, Siddiqui AH, Veznedaroglu E, Turner RD, Ringer AJ, Mocco J. Simulator based angiography education in neurosurgery: results of a pilot educational program. J Neurointerv Surg. 2012;4(6):438–441.
16. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1999;177(1):28–32.
17. Hsu JH, Younan D, Pandalai S, et al.. Use of computer simulation for determining endovascular skill levels in a carotid stenting model. J Vasc Surg. 2004;40(6):1118–1125.
18. Van Herzeele I, Aggarwal R, Neequaye S, et al.. Experienced endovascular interventionalists objectively improve their skills by attending carotid artery stent training courses. Eur J Vasc Endovasc Surg. 2008;35(5):541–550.
19. Dayal R, Faries PL, Lin SC, et al.. Computer simulation as a component of catheter-based training. J Vasc Surg. 2004;40(6):1112–1117.
20. Spiotta AM, Rasmussen PA, Masaryk TJ, Benzel EC, Schlenk R. Simulated diagnostic cerebral angiography in neurosurgical training: a pilot program [published online ahead of print May 10, 2012]. J Neurointerv Surg. doi:10.1136/neurintsurg-2012-010319.
21. Schreiber TL, Strickman N, Davis T, et al.. Carotid artery stenting with emboli protection surveillance study: outcomes at 1 year. J Am Coll Cardiol. 2010;56(1):49–57.
22. Willaert W, Aggarwal R, Harvey K, et al.. Efficient implementation of patient-specific simulated rehearsal for the carotid artery stenting procedure: part-task rehearsal. Eur J Vasc Endovasc Surg. 2011;42(2):158–166.
23. Willaert WI, Aggarwal R, Daruwalla F, et al.. Simulated procedure rehearsal is more effective than a preoperative generic warm-up for endovascular procedures. Ann Surg 2012;255(6):1184–1189.
24. Willaert WI, Aggarwal R, Van Herzeele I, Cheshire NJ, Vermassen FE. Recent advancements in medical simulation: patient-specific virtual reality simulation. World J Surg. 2012;36(7):1703–1712.
25. Hislop SJ, Hedrick JH, Singh MJ, et al.. Simulation case rehearsals for carotid artery stenting. Eur J Vasc Endovasc Surg. 2009;38(6):750–754.
26. Dawson DL, Lee ES, Hedayati N, Pevec WC. Four-year experience with a regional program providing simulation-based endovascular training for vascular surgery fellows. J Surg Educ. 2009;66(6):330–335.
27. Azuma RT. A survey of augmented reality. Presence: Teleoperators & Virtual Environments. 1997;6(4):355–385.
28. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality. IEEE Comput Graph Appl. 2001;21:34–47.
29. Macchiarella ND, Liu D, Gangadharan SN, Vincenzi DA, Majoros AE. Augmented Reality as a Training Medium for Aviation/Aerospace Application. Thousand Oaks, CA: SAGE Publications; 2005:2174–2178.
30. Neumann U, Majoros A. Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. Proc. IEEE Virtual Reality Annual International Symp. (VRAIS 98). Los Alamitos, CA: IEEE CS Press, 1998:4–11.
31. Rios H, Hincapié M, Caponio A, Mercado E, González Mendívil E. Augmented reality: an advantageous option for complex training and maintenance operations in aeronautic related processes. In: Virtual and Mixed Reality: New Trends. New York/Heidelberg: Springer, 2011:87–96.
32. Macchiarella ND, Liu D, Vincenzi DA. Q augmented reality as a means of job task training in aviation. In: Hancock PA, Vincenzi DA, Wise JA, Mouloua M, eds. Human Factors in Simulation and Training. 1st ed. Boca Raton, FL: CRC Press; 2008:333.
33. Söderman M, Babic D, Homan R, Andersson T. 3D roadmap in neuroangiography: technique and clinical interest. Neuroradiology. 2005;47(10):735–740.
34. Racadio JM, Babic D, Homan R, et al.. Live 3D guidance in the interventional radiology suite. AJR Am J Roentgenol. 2007;189(6):W357–W364.
35. Okumura H, Terada T, Babic D, Homan R, Katsuma T. 3D Roadmapping in neuroendovascular procedures: an evaluation. MedicaMundi. 2010;54(3):5–11.
36. Abe T, Hirohata M, Tanaka N, et al.. Clinical benefits of rotational 3D angiography in endovascular treatment of ruptured cerebral aneurysm. AJNR Am J Neuroradiol. 2002;23(4):686–688.
37. Cooke DL, Levitt M, Kim LJ, Hallam DK, Ghodke B. Intraorbital access using fluoroscopic flat panel detector CT navigation and three-dimensional MRI overlay. J Neurointerv Surg. 2010;2(3):249–251.
38. Cooke DL, Levitt M, Kim LJ, Hallam DK, Ghodke B. Transcranial access using fluoroscopic flat panel detector CT navigation. AJNR Am J Neuroradiol. 2011;32(4):E69–E70.
39. Saeed M, Hetts SW, English J, Wilson M. MR fluoroscopy in vascular and cardiac interventions (review). Int J Cardiovasc Imaging. 2012;28(1):117–137.
40. Ruijters D, Homan R, Mielekamp P, van de Haar P, Babic D. Validation of 3D multimodality roadmapping in interventional neuroradiology. Phys Med Biol. 2011;56(16):5335–5354.