Sutherland, Garnette R. MD*; Wolfsberger, Stefan MD*,‡; Lama, Sanju MD*; Zarei-nia, Kourosh PhD*
NeuroArm evolved as a potential method for improving the integration of imaging with microsurgery and stereotaxy.1,2 neuroArm has the potential to perform surgery within the bore of a high-field magnet, which would allow high-quality intraoperative imaging without interrupting the rhythm of surgery.3 Surgeons will be able to see and manipulate imaging data from the remote workstation of neuroArm without compromising sterility or unduly prolonging the surgical procedure. neuroArm was constructed to grow alongside developments in the field of neurosurgery; for instance, it is well-suited to the growing trend of surgical minimalism.
neuroArm is a teleoperated magnetic resonance (MR)-compatible image-guided robot (Figure 1). The neuroArm system comprises 2 robotic arms capable of manipulating both specially designed and existing microsurgical tools connected via a main system controller to a workstation with a sensory immersive human-machine interface. The surgeon is positioned at a workstation and uses the human-machine interface to interact with the surgical site. The human-machine interface provides both MR images (MRIs) and real-time high-definition 3-dimensional (3-D) images of the surgical site. Modified PHANTOM hand controllers (Wilmington, Massachusetts) enable the manipulators to emulate the surgeon’s hand movements while providing the surgeon with haptic force feedback.4 The modification includes a stylus gimbal that enables the measurement of an additional 3 degrees of positional sensing (pitch, roll, and yaw). The displacement of the hand controller is mimicked by the robot manipulators. The interaction force between the surgical tool and the environment (eg, brain tissue), measured by 2 force sensors mounted on each arm, is applied to the surgeon’s hand by the haptic device.5 It has been shown that, in some applications of teleoperation systems, haptic feedback alone can be more valuable than visual feedback alone.6 neuroArm operates in conjunction with a surgical assistant present at the surgical site (Figure 2). All members of the surgical team are equipped with communication headsets. The neuroArm system allows the surgeon to access imaging data without interrupting surgical procedure while providing the surgeon with the most technologically advanced sensory input available to best facilitate the performance of surgery.
Operating within real-time images means that surgeons can correct for brain shift and can ensure the complete removal of a tumor during operation. Currently, MR is the imaging technology of choice for neurosurgery because of its high soft-tissue resolution, but with imaging-compatible robots, there is the developing possibility of fusing image sets to supply the benefits of multiple imaging technologies. neuroArm is moving toward the fusion of image sets through its real-time tool set for intraoperative neurosurgical assessment. For instance, diagnostic images, including computerized tomography, contrast angiography, ultrasound, positron emission tomography, and MR, could be registered with the robot navigation system to combine with the updated intraoperative MRI data. This would allow surgeons to see information related to brain anatomy, function, and metabolism during surgery. This will undoubtedly lead to further improvements in precision and accuracy.
The neuroArm project began in September 2001. The research team based at the University of Calgary investigated other surgical robots under development and approached several potential collaborators. MacDonald, Dettwiler and Associates Ltd (MDA), a company that specializes in building robotics for complicated environments, including the Shuttle Remote Manipulator System (colloquially known as Canadarm), responded positively to the opportunity to develop a robot for neurosurgery. As a result of the association with a company that had already developed a great deal of teleoperated robotic technology, neuroArm benefited from the inclusion of knowledge related to the prior aerospace achievements of MDA. Stability and transparency in bilateral haptic teleoperation of robotic arms have been a challenge during the last 2 decades.7 First, the overall closed-loop system should be stable regardless of the operator’s behavior or the task environment.5 Second, the surgeon must be able to feel the interaction of the robot at the surgical site to accomplish the task effectively. This is technically achieved if position tracking at the robot site and force tracking at the surgeon site are both faithful.8
STEPS TOWARD THE MANUFACTURE OF NEUROARM
First, a document outlining the project requirements was compiled (Figure 3). Engineers from MDA were brought into the operating room to observe the surgeons in their environment over the course of several weeks to witness several different types of pathology and to project how the robot would integrate into the operating room. Based on this requirements document, the overall feasibility of the project was analyzed, and a preliminary design review determined that microsurgery would need to be decoupled from the magnet because vision technology was not yet advanced enough and components were too large to achieve microsurgery within the bore of the magnet. The investigators decided that constructing a component of the project would be an essential first step toward their ultimate vision, so the project continued with an altered scope. At this stage, mechanical and electrical design and manufacturing issues related to the interference of the high magnetic field inside the bore were solved.
Next, a critical design review determined the feasibility of constructing the robot according to regulatory requirements. This stage consists of an intensive review of the entirety of the design. It is the critical design review that demonstrates whether the project can be completed. At this point, contract renegotiation is common because the viability of the project has been established and the scope may have changed from the initial stages of planning. In this case, the intellectual property (IP) remained with the University of Calgary, so the cost of the project rose because the value of the IP had been proven. The contract negotiation took 1 year. The cost increased from $6.5 to $10.5 million, and the contract fixed both cost and time tied to a detailed statement of work.
After the critical design review was completed, the manufacture and testing of neuroArm began. During the manufacture of neuroArm, weekly teleconferences between the medical researchers at the University of Calgary and the engineering team at MDA kept everyone informed about the development process and allowed minor modifications to the design. This ongoing communication cemented the teamwork between the physicians and engineers.
In the time between the preliminary design review and the critical design review, a company was established, neuroArm Surgical, to hold the neuroArm IP. The company had an advisory board that consisted of some community business leaders, an accountant, a lawyer, a university representative, and the project leader. The advisory board members had experience dealing with the business side of project management and provided stability to the project when difficulties arose. The company hired a chief executive officer who was responsible for managing the project IP, share structure, and fundraising. The purpose of the company was to create value for the product through IP protection, paving the way for future commercialization.
Once neuroArm was manufactured and delivered, it had to be integrated into the operating room. This involved a signoff process in which the engineers systematically demonstrated all of the robot components to prove that they met the specifications agreed on in the requirements document. This extensive documentation also expedited Health Canada regulatory approval because it demonstrated that the robot was built to appropriate standards and provided the traceability of all components.
Institutional ethical and regulatory approval depends on extensive documentation. Before the robot is implemented in neurosurgery, it must meet electrical, institutional, and federal requirements. For preclinical testing, institutional ethics approval is required. To reduce the waiting period between approvals and clinical testing, the regulatory application was submitted simultaneously to the University of Calgary ethics board and to Health Canada. Health Canada’s initial approval for neuroArm was provisional on the approval of the University of Calgary’s ethics board. The contract between the medical researchers and MDA stipulated that neuroArm must be built to Food and Drug Administration standards. Building to a high international regulatory standard eases the approvals process and ensures a high level of robot safety.
A virtual trainer was built to accustom the surgeon to the hand controllers of neuroArm (Figure 4). This trainer includes tests that would emulate the skills needed for suturing, biopsy, and microsurgery.9 Operating the hand controller in conjunction with the screen is an important skill to practice before using neuroArm in a preclinical situation because it takes some time for the surgeon to acclimatize to the way the hand controllers relate to the identical motion of the robotic manipulators, the dimensions of the workspace, and positioning the manipulators in relation to that workspace. The virtual trainer emulates how the robotic manipulators respond to commands issued by the surgeon through the haptic hand controllers. It also helps the surgeon get used to the virtual forces generated by the haptic hand controllers and provides the surgeon with experience interpreting and relating these forces to the sensation of touch in conventional surgery.
Additional models were prepared to help the surgeon adjust to using the workstation to manipulate neuroArm in real space. One such model had the surgeon dropping washers onto a pegboard using the neuroArm systems. Another consisted of a pick-and-place task using neuroArm to move cotton balls from 1 bowl into another. These are typical tasks usually used for training and performance evaluation of haptic teleoperation systems.10,11 Through these models, the surgeon gained familiarity and comfort with the software, hand controllers, tools, command and status display graphic user interface, and 3-D monitors of neuroArm. As a result, the surgeon’s precision, accuracy, and speed with neuroArm improved before preclinical testing.
Once institutional ethics approval had been received, preclinical testing began. Experiments were performed on rat models, including splenectomy, bilateral nephrectomy, and removal of the submandibular gland.3 These procedures were performed both with and without the robot for comparison. These microsurgical procedures allowed testing for hemostasis, precision, accuracy, and dexterity.
Cadaver testing allowed neuroArm to operate within the bore of the magnet.3 This image-guided surgery in a 3-D environment allowed the surgeon to learn how to interact with the MRI display, to set no-go zones, and to hit targets with precision and accuracy, thus familiarizing the surgeon with the navigational capabilities of the robot. No-go zones use the concept of virtual fixtures, which limit the movements of the manipulator to constrained regions or along predefined paths within the workspace. The use of virtual fixtures can dramatically increase the level of safety and precision of a robot; virtual fixtures can additionally be used for guidance and navigation purposes.12
neuroArm was introduced into surgery in a graded fashion to account for the multiple variables a surgical robot introduces to the operating room (Figure 5). neuroArm has been used in 35 cases. Each patient provided informed consent, and procedures took place at the Foothills Medical Centre, Calgary, Alberta, Canada. All procedures were performed in compliance with the University of Calgary Conjoint Health Research Ethics Board of the Faculties of Medicine, Nursing, and Kinesiology, as well as with Health Canada guidelines.
Surgical exposure, including craniotomy and dura opening, was performed using conventional technique. During the opening, the robotic arms were draped, and sterilized components for holding and actuating surgical instruments were inserted through the drape. Surgical tools, selected from bipolar forceps, tissue forceps, a needle driver, tissue dissectors, suction, and microscissors, were placed into the right and left manipulator. The surgical microscope, modified with 2 high-definition cameras, provided stereoscopic vision of the surgical site. The cameras are Sony PMW-10MD full high-definition medical-grade cameras equipped with three ½-in (1920 × 1080) Exmor CMOS sensors, each delivering > 2 million pixels. Images are then displayed on an LMD-2451MT high-definition medical monitor that enables surgeons to gain full depth perception and spatial orientation during intricate procedures through clear 3-D picture display. Surgeons wear light, comfortable polarized glasses to view the 3-D display. These polarized glasses are almost clear and do not interfere with viewing the other workstation displays.
neuroArm was positioned as the primary surgeon for microsurgical dissection. The neurosurgical resident assumed the role of surgical assistant for all cases. In some cases, neuroArm was registered to the interoperatively acquired images after image acquisition but before draping. Routine dissection of the brain-tumor interface has occurred only over the last 15 cases because it took the surgeon approximately 20 cases to become comfortable with and confident in the use of the robot while accounting for variations in pathologies.
After the first 5 cases, which were imaged with a 1.5-T magnet, the operating room was upgraded to a 3.0-T system. This upgrade began in June 2008, and use of neuroArm was halted until its completion in October 2010.13 During this time, an engineering safety review was performed in response to an adverse event in case 5. The patient was unharmed. In this case, a translabyrinthine exposure was performed for excision of a recurrent prepontine dermoid. The tumor capsule was coagulated, and the lesion was entered with neuroArm. The dermoid contents were aspirated with the left manipulator suction tool. During this stage, the left manipulator moved to the left when commanded to move to the right, which caused the suction tool to break when it hit a retractor. The remainder of the procedure was accomplished with the use of only the right manipulator with the surgical assistant completing the aspiration of dermoid contents. This unexpected movement triggered the comprehensive safety review. The safety review generated multiple new safety features, including an emergency stop switch that bypasses the computer so that the surgeon can instantaneously halt the robot at any time, and no-go zones that prevent tools from leaving a designated corridor. There has been no further incidence of unintended motion.
Coincidently, case 5 was filmed by the BBC, and an episode of SuperDoctors called “Robot Surgeons” features footage from this case (http://www.bbc.co.uk/programmes/b00d4nx7). It seems a bit unfortunate that of all the cases neuroArm has performed, such a prestigious news organization happened to be filming during the only case in which the robot experienced an adverse event. However, this demonstrated the breadth of the project because the surgeons were able to complete the task despite the unexpected complication, and this could serve to ease some of the public’s discomfort about how surgeons introduce new technologies into the operating room.
In another case, the use of neuroArm was halted in favor of conventional surgical procedure when the surgical corridor proved too restricted for the manipulators and surgical assistant to access at the same time. The challenge of corridors that restrict access to only 1 surgeon also exists in conventional surgery. Although this is currently a shortcoming, as the size of components decreases, it will become possible for neuroArm to operate alongside a surgeon in environments in which positioning would make it difficult for 2 human surgeons to access the surgical site. In no other case did neuroArm disrupt established neurosurgical, nursing, or anesthetic practices.
One of the most unique features of neuroArm is its sensory immersive workstation.2 This workstation allows the surgeon to access the surgical site remotely and provides some of the sensations of surgery through visual, aural, and haptic technologies. Haptics is an expanding field that focuses on replicating human touch. The discrepancy between human touch and the sensations provided by haptic technology is significant, but the field of haptics is constantly incorporating more complex understandings of human touch into new products. Haptics technology is taking advantage of different mechanical and electrical developments to apply forces, vibrations, motions, or even weak electric shocks to provide a virtual sense of touch.14 Sensory technologies are rapidly improving owing to growth in consumer electronics, which means that the 3-D monitors are providing crisper pictures, haptic devices are providing force/tactile feedback that is closer to human touch, and the communication headsets are allowing clearer sound. The surgeon operating neuroArm found that it was easier to take full advantage of the assistant surgeon’s skills while using the robot than in conventional surgery because of the positioning of the robot opposite the surgical assistant, resulting in 2 surgeons constantly engaged in the surgical procedure. This combines the precision and accuracy of the robot with the dexterity and maneuverability of the human surgeon.
neuroArm has electronic tremor filters that enable smooth movement of the manipulators. These filters could potentially be set to the specific frequency of a personal tremor and could have variable settings to compensate for different tremors that a surgeon experiences, although they are currently used on a general setting. These tremor filters benefit the precision and accuracy of surgery.
Motion scaling allows the robot to move on a different scale than the surgeon. Currently, motion scaling goes from 1/1 to 1/20, allowing the surgeon to make bigger gestures and the machine to adjust the movement to the target size. This enables surgeons to perform movements that would not otherwise be feasible because of the small scale. Operators can fine-tune the scaling variables to adjust for both the demands of the surgery and their own comfort level, much like the resolution gains of the mouse of a personal computer.15 Speed can also be scaled, allowing the tool to move more slowly than the surgeon’s hand, which creates a smoother motion.
The present encoders are accurate to within 0.01°. This precision is coupled with high-performance gears. neuroArm is capable of moving a 500-g payload at 200 mm/s to a target with 1-mm accuracy.
Another feature of neuroArm is the z lock, which enables the manipulator to move in a straight line toward a target, even if the surgeon does not keep the hand controllers moving straight. The z-lock feature of neuroArm uses the concept of guidance virtual fixtures that help to keep the manipulator on a desired surface or path.12 The precision and accuracy of neuroArm are expected to improve with each generation. neuroArm II is currently in development and will be able to operate on a finer level than neuroArm I owing to advanced encoders, improved joints, gears manufactured from plastic polymers, and upgraded software.
The workstation has the potential to control all operating room technologies and eventually to connect the surgeon to the Internet to access global medical knowledge or to communicate directly with experts at other institutions during surgery. Additionally, the workstation can be used for case rehearsal, which has positive implications for surgeon training, and will develop personalized case cards, making surgical preparations more efficient. Surgeons will be able to gain experience from a virtual environment. Surgeries could be recorded and replayed. All elements of surgery will become measurable; eg, the amount of pressure a surgeon needs to use to cut through a particular type of tissue will be quantifiable, and surgeons in training will be able to practice exerting the precise amount of pressure necessary with the workstation providing instructional feedback. The workstation empowers the surgeon to prepare for all contingencies.
Currently, only 1 neuroArm manipulator can fit into the bore of the magnet. This necessitated decoupling biopsy from microsurgery. neuroArm II includes new materials and has upgraded the encoders, resulting in a robot that is at least 20% smaller and has less effect on the MR signal-to-noise ratio, creating a better image. The smaller components and new joint design of neuroArm II will be able to fit both manipulators into the bore of the magnet, expanding surgical capabilities. The new encoders provide more accuracy. The updated joint architecture is based on the Canadarm joints and allows increased payload. The workstation has been streamlined for ease of use.
With mechanical improvements in precision and accuracy, there will be a movement toward true microsurgery, ie, surgery on the cellular level. neuroArm I operates at 50 μm, and neuroArm II will be capable of operating at 20 to 30 μm. Improvements in technological components are progressing toward finer levels of operation. Robotics can be used to enhance the abilities of human surgeons, eventually allowing cellular manipulation within the surgical corridor. Through advanced human-machine interface, surgeons will be able to see, hear, and touch things on a scale that is beyond unaugmented human capacity, progressing surgery from the organ to the cellular level.16,17 Advances in molecular imaging, the imaging of normal and abnormal cells and genes, and innovations in robotic technology combined will bring cellular surgery into view on the horizon of medical development.
This work was supported by grants from the Canada Foundation for Innovation, Western Economic Diversification, Canada, and Alberta Advanced Education and Technology. Dr Sutherland holds shares in IMRIS, the Canadian company manufacturing and distributing both intraoperative MRI and neuroArm technology. His name is listed on many of the founding patents of IMRIS. The other authors have no personal financial or institutional interest in any of the drugs, materials, or devices described in this article.
We would like to thank the engineers at MDA for their collaboration on the design and manufacture of neuroArm. Additional thanks are extended to the advisory board of neuroArm Surgical Ltd for assistance with the neuroArm project, as well as to IMRIS for ongoing efforts in the design and manufacture of neuroArm II. Appreciation is also extended to Claire Lacey, BA, MA, for her considerable help in preparing this manuscript.
1. Sutherland GR, Newhook P, Feil G, Fielding T, Greer AD, Latour I. An image-guided MR compatible surgical robot. Neurosurgery. 2008;62:286–293.
2. Greer AD, Newhook P, Sutherland GR. Human-machine interface for robotic surgery and stereotaxy. IEEE/ASME Trans MRI Compatible Mechatronic Syst. 2008;13(3):355–361.
3. Pandya S, Motkoski JW, Serrano-Almeida C, Greer AD, Latour I, Sutherland GR. Advancing neurosurgery with image-guided robotics. J Neurosurg. 2009;111:1141–1149.
4. Massie TH, Salisbury J. The phantom haptic interface: a device for probing virtual objects. J Dyn Syst Meas Control. 1994;55(1):295–299.
5. Hokayem PF, Spong MW. Bilateral teleoperation: an historical survey. Automatica. 2006;42(12):2035–2057.
6. Ming O, Beard DV, Brooks FP Jr. Force display performs better than visual display in simple 6-D docking task. Paper presented at: IEEE International Conference on Robotics and Automation; May 14-19, 1989; Scottsdale, AZ.
7. Zarei-nia K, Sepehri N, Wu Q. A Lyapunov controller for stable haptic manipulation of hydraulic actuators. Int J Robust Nonlinear Control. 2012;22(3):241–261.
8. Zarei-nia K, Yazdanpanah GA, Sepehri N. Experimental evaluation of bilateral control schemes applied to hydraulic actuators: a comparative study. Trans Can Soc Mech Eng. 2009;33(3):377–398.
9. Choi MY, Sutherland GR. Surgical performance in a virtual environment. On the Horizon. 2009;17(4):345–355.
10. Ziherl J, Podobnik J, Sikic M, Munih M. Pick to place trajectories in human arm training environment. Technol Health Care. 2009;17(4):323–335.
11. Unger B, Nicolaidis A, Berkelman P, Thompson A, Klatzky R, Hollis R. Comparison of 3D haptic peg-in-hole tasks in real and virtual environments. Paper presented at: Proc. Int'l Conf. on Intelligent Robots and Systems (IROS '01), Maui, Hawaii, October 29-November 3, 2001.
12. Zarei-nia K, Sepehri N, Olson T, Mueller W. Haptic-enabled control of hydraulic manipulators applied to power line maintenance: concept implementation. Paper presented at: 1st International Conference on Applied Robotics for the Power Industry; October 5-7, 2010; Montreal, QC, Canada.
13. Lang MJ, Kelly JJP, Sutherland GR. A moveable 3-Tesla intra-operative MRI system. Neurosurgery. 2011;68(ONS suppl 1):ons168–ons179.
14. El Saddik A, Orozco M, Eid M, Cha J, eds. Haptics Technologies: Bringing Touch to Multimedia. Ottawa, Canada: Springer; 2011.
15. Kosuge K, Itoh T, Fukuda T. Scaled telemanipulation with communication time delay. IEEE Int Conf Robotics Automation. 1996;3:2019–2024.
16. Iqbal U, Trojahn U, Albaghdadi H, et al.. Kinetic analysis of novel mono- and multivalent VHH-fragments and their application for molecular imaging of brain tumors. Br J Pharmacol. 2010;160(4):1016–1028.
17. Tomanek B, Iqbal U, Blasiak B, et al.. Evaluation of brain tumor vessels specific contrast agents for glioblastoma imaging. Neuro Oncol. 2012;14(1):53–63.