Innovations: Technology & Techniques in Cardiothoracic & Vascular Surgery:
Augmented Reality Image Guidance Improves Navigation for Beating Heart Mitral Valve Repair
Chu, Michael W.A. MD*†‡; Moore, John MA, MSc†; Peters, Terry PhD, FIEEE†§; Bainbridge, Daniel MD†∥; McCarty, David MB, BCh, MRCP¶; Guiraudon, Gerard M. MD†‡; Wedlake, Chris BSc†; Lang, Pencilla BEng*†; Rajchl, Martin MSc†; Currie, Maria E. MD*†; Daly, Richard C. MD#; Kiaii, Bob MD*†‡
From the *Division of Cardiac Surgery, Department of Surgery, †Robarts Research Institute, Western University; ‡Canadian Surgical Technologies and Advanced Robotics; §Departments of Medical Imaging, ∥Anaesthesia, ¶Division of Cardiology, Department of Medicine, Western University, London, ON Canada; and #Mayo Clinic, Rochester, MN USA.
Accepted for publication August 29, 2012.
Funding for this work was provided by Canadian Foundation for Innovation (20994), the Ontario Research Fund (RE-02-038), and the Canadian Institutes of Health Research (179298). Funding for the animal laboratory study was received from NeoChord, Inc., Eden Prairie, MN USA.
Presented the Young Investigator Award at the Annual Scientific Meeting of the International Society for Minimally Invasive Cardiothoracic Surgery, May 30–June 2, 2012, Los Angeles, CA USA.
Disclosures: Gerard Guiraudon has a licensing agreement for GUCI, with Valve Xchange Inc., in Aurora, Denver, CO USA, and is a Special Medical Advisor with the company. Richard C. Daly, MD, is a consultant at NeoChord, Inc., Eden Prairie, MN USA. Bob B. Kiaii, MD, is a consultant at Medtronic, Inc., Minneapolis, MN USA. John Moore, MA, MSc, Terry Peters, PhD, FIEEE, Daniel Bainbridge, MD, David McCarty, MB, BCh, MRCP, Gerard M. Guiraudon, MD, Chris Wedlake, BSc, Pencilla Lang, BEng, Martin Rajchl, MSc, Maria E. Currie, MD, declare no conflict of interest.
Address correspondence and reprint requests to Michael W.A. Chu, MD, B6-106 University Hospital, London Health Sciences Centre, 339 Windermere Rd, PO Box 5339, London, ON Canada N6A 5A5. E-mail: email@example.com.
Objective: Emerging off-pump beating heart valve repair techniques offer patients less invasive alternatives for mitral valve (MV) repair. However, most of these techniques rely on the limited spatial and temporal resolution of transesophageal echocardiography (TEE) alone, which can make tool visualization and guidance challenging.
Methods: Using a magnetic tracking system and integrated sensors, we created an augmented reality (AR) environment displaying virtual representations of important intracardiac landmarks registered to biplane TEE imaging. In a porcine model, we evaluated the AR guidance system versus TEE alone using the transapically delivered NeoChord DS1000 system to perform MV repair with chordal reconstruction.
Results: Successful tool navigation from left ventricular apex to MV leaflet was achieved in 12 of 12 and 9 of 12 (P = 0.2) attempts with AR imaging and TEE alone, respectively. The distance errors of the tracked tool tip from the intended midline trajectory (5.2 ± 2.4 mm vs 16.8 ± 10.9 mm, P = 0.003), navigation times (16.7 ± 8.0 seconds vs 92.0 ± 84.5 seconds, P = 0.004), and total path lengths (225.2 ± 120.3 mm vs 1128.9 ± 931.1 mm, P = 0.003) were significantly shorter in the AR-guided trials compared with navigation with TEE alone. Furthermore, the potential for injury to other intracardiac structures was nearly 40-fold lower when using the AR imaging for tool navigation. The AR guidance also seemed to shorten the learning curve for novice surgeons.
Conclusions: Augmented reality–enhanced TEE facilitates more direct and safe intracardiac navigation of the NeoChord DS tool from left ventricular apex to MV leaflet. Tracked tool path results demonstrate fourfold improved accuracy, fivefold shorter navigation times, and overall improved safety with AR imaging guidance.
Degenerative mitral valve (MV) disease affects 2% of the general population1 and commonly results in leaflet prolapse, chordal rupture, and mitral regurgitation and leads to left ventricular (LV) dysfunction.1,2 Mitral valve repair can relieve symptoms, reduce morbidity, and restore life expectancy.1–3 Resection techniques of flail and prolapsing MV leaflet segments have been the gold standard therapy for MV repair4; however, valve-preserving techniques with neochordae reconstruction have emerged with equally proficient repair results.5–7 In fact, some evidence suggests that implantation of neochordae may result in superior hemodynamics and a more durable repair as a result of the longer height of leaflet coaptation.8 Chordal reconstructive techniques are versatile as they can be applied to a wide variety of leaflet prolapse, including anterior, posterior, and commissural leaflet involvement.
Recently, many novel transcatheter valve techniques have been developed to enable beating heart MV repair.9–18 The NeoChord DS1000 (NeoChord Inc., Eden Prairie, MN USA) is an off-pump device that uses transapical access to deliver expanded polytetrafluoroethylene neochordae to flail segments of the MV.16–18 This procedure is in early clinical trials and currently relies on transesophageal echocardiography (TEE) guidance alone for tool navigation and neochordae deployment. Although TEE guidance has been good for final positioning of the tool and grasping the leaflet, there can be recognized difficulties in navigating the tool in the shortest, most direct path from the LV apex to the left atrium. Transesophageal echocardiographic navigation lacks the spatial and temporal resolution to view the LV apex, the tool tip, and the intended MV leaflet segment simultaneously; therefore, risking potential tool injury of the aortic valve, LV free wall, or subvalvar MV apparatus or leaflet perforation.
We believed that an augmented reality (AR) image guidance system will help reduce these risks and improve procedural navigation and safety.
All experiments were conducted under the supervision of a veterinarian and with the approval of the Animal Use Subcommittee of the University Council on Animal Care at Western University, London, ON Canada. A total of six cardiovascular surgeons participated in the study, performing the neochordae image navigation tasks a total of 12 times with and without AR imaging guidance in two porcine models. The primary goal of the study was to compare the navigation of the NeoChord DS1000 tool from the LV apex across the mitral orifice to the target MV leaflet region with and without AR assistance. The NeoChord tool location was recorded every 0.5 seconds for all navigation attempts. Four metrics were used to assess the navigation task including (i) the total length the tool path followed; (ii) the distance error from tool locations to a direct line drawn from apex to final destination in the coaptation line; (iii) total task completion time; and finally, (iv) the tool path data were used to identify points of potential injury to neighboring intracardiac structures. Successful navigation was determined by identification of the tool facing the appropriate MV leaflet segment, as indicated by TEE.
All animals underwent general anesthesia with single-lumen endotracheal tube intubation and monitoring using a right internal jugular venous line, femoral arterial line, three-lead electrocardiography, oxygen saturation probe, and end-tidal carbon dioxide monitoring. A lower hemisternotomy was performed, and pericardial retention sutures were placed to expose the LV apex. Two plegetted 2-0 Prolene purse-string sutures were placed around the LV apex with Rummel tourniquets to secure the apical puncture site. The NeoChord DS1000 device was introduced into LV apex. The surgeon then used TEE guidance alone or AR imaging guidance to navigate the tool from the LV apex across the MV orifice to face the open tool jaw toward the predetermined intended MV leaflet segment. After the image guidance study, the targeted MV leaflet segment was grasped by the jaws of the device. Correct leaflet capture was verified using the fiber optic–based detection mechanism that is integrated into the NeoChord DS1000 tool. After leaflet capture was verified, an expanded polytetrafluoroethylene suture was pulled through the leaflet and the tool was retracted out the LV apex with both ends of the suture. The suture was fixed at the leaflet with a girth hitch knot, adjusted under real-time TEE guidance to the appropriate length, and then secured at the apex using a pledget.
TEE Guidance Alone, No AR
After tool entry into the LV apex, two-dimensional biplane TEE was used to identify the tool and carefully navigate it across the subvalvar apparatus and just through the MV orifice. We attempted to maintain the tool tip, tool profile, and final target site in the echo image planes at all times. Correct positioning and orientation of the tool jaws were then achieved using three-dimensional TEE.
AR Echocardiographic Guidance
We have developed a visualization environment19 that uses tracking technology to locate both the tool and the TEE probe in three-dimensional space, making it possible to represent the real-time echo images with virtual representations of both devices and interactively defined intracardiac anatomy within a common coordinate system (Fig. 1). Sensors from the Aurora (Northern Digital, Waterloo, ON Canada) magnetic tracking system were integrated inside the NeoChord tool (Fig. 2A) and onto the TEE probe of the Philips iE33 ultrasound (Fig. 2B). Virtual representations of the TEE probe and NeoChord tool were created in Visualization Toolkit, and the tools were appropriately calibrated.20 Axes with 10-mm markings were projected from the virtual representation of the NeoChord DS1000, indicating the forward trajectory of the tool and the direction of the opening jaws to facilitate planning the intended tool trajectory from the LV apex toward the desired MV leaflet target site. In addition, tracking the TEE image data made it possible to define aortic annulus, mitral annulus, and mitral line of coaptation for contextual purposes (Fig. 1). These features were defined in midsystole before the introduction of the NeoChord tool into the heart. As a result, we hoped that our AR guidance system would assist with three navigation tasks: (i) planning the LV access point and trajectory, (ii) maintaining a safe and direct entry through the subvalvar apparatus across the line of coaptation, and (iii) establishing the correct tool orientation to grasp the appropriate MV segment. Biplane TEE images were created by digitally frame-grabbing the images from the Philips iE33 TEE system and mapping them onto a pair of intersecting planes that can be positioned in a three-dimensional visualization environment in which the NeoChord instrument and the targets are defined (Fig. 1).
The technology associated with the AR navigation system has minimal impact in the operating room. The Aurora Tabletop magnetic field generator (Northern Digital, Waterloo, ON Canada) is specifically designed to operate in the presence of ferromagnetic objects below the plane of the generator, which has a large field of view, and easily fits under the patient padding on the OR table (Fig. 2C). It does not interfere with access to the patient. Sensors attached to the TEE probe and surgical tools are designed such that they do not impede normal operating room workflow (Fig. 2D).
Continuous variables were expressed as a mean ± SD, and categorical variables were expressed as percentages. Statistical analysis was performed using paired t test and Fisher exact test for dichotomous variables and a value of P < 0.05 was considered statistically significant.
Successful tool navigation from LV apex to MV leaflet was achieved in 12 (100%) of 12 and 9 (75%) of 12 (P = 0.2) attempts with AR imaging and TEE alone, respectively. The AR imaging guidance demonstrated significantly shorter distance errors from midline, total navigation times, and total path lengths traveled compared with TEE navigation alone (Table 1). The AR imaging guidance allowed for more accurate tool navigation from LV apex to the MV leaflet with greater than threefold reduction in distance error from the intended midline trajectory (P = 0.003) and greater than fourfold reduction in total tracked tool path lengths before reaching the intended MV leaflet target (P = 0.003) when compared with TEE guidance alone. The total navigation times for the tool within the LV was reduced fivefold with AR imaging compared with TEE alone (P = 0.004). Figures 3 and 4 demonstrate the striking differences in the magnetically tracked tool paths taken by each surgeon (each surgeon represents a different color) when using TEE guidance alone versus AR imaging guidance. We also analyzed these results by individual surgeon study, and Figure 5 demonstrates that even a surgeon with multiple NeoChord DS1000 human implants (one of the original inventors) can derive significantly improved tool navigation accuracy with AR guidance versus TEE alone.
Although tool navigation accuracy is paramount, equally important is patient safety and prevention of iatrogenic injury to the aortic valve, anterior and posterior leaflets, LV free wall, and left atrial roof. Table 2 demonstrates the number of times the tool traversed other intracardiac structures with TEE guidance alone versus AR imaging guidance. In total, there were 78 instances of potential injury using TEE alone compared with only 2 in the trials using the AR imaging guidance system, resulting in an approximate 40-fold decrease in risk of possible injurious events (P = 0.008).
When comparing navigation outcomes between novices and experts (Table 3, Fig. 6), there were no statistically significant differences between the two groups. However, the confidence intervals narrow significantly with AR imaging versus TEE imaging alone, suggesting that AR imaging improves the reproducibility of the procedure and may help novice surgeons to more expeditiously overcome the steep learning curve associated with navigating the NeoChord DS1000 tool (Fig. 6).
Emerging transcatheter valve therapies have introduced various MV repair strategies on the beating heart, including edge-to-edge repair,9–11 coronary sinus cerclage,12,13 transcatheter annuloplasty,14 and neochordal reconstruction.15–18 These experimental techniques avoid the deleterious effects of cardiopulmonary bypass but continue to be hampered by limited intraoperative visualization and suboptimal procedural accuracy. All of these transcatheter MV repair techniques rely on single-plane fluoroscopy and/or TEE. Intraoperative TEE provides good real-time visualization of MV anatomy but struggles to provide adequate navigation and guidance of surgical tools. Intraoperative fluoroscopy (with and without contrast) provides reasonable visualization of surgical tools, but often lacks three-dimensional context with only gross anatomic structures visible. As a result, current imaging modalities provide limited intracardiac views that make transcatheter manipulation of the MV challenging and can result in prolonged procedural times, additional device deployments, excessive radiation exposure, and most importantly, increased complication rates.18,21 Although many advances have occurred in transcatheter valve therapy designs, there remains great dependence on current imaging modalities and their known limitations.21
A variety of AR imaging guidance systems have been developed to facilitate intracardiac beating heart surgery.19,20,22,23 We believe that our AR imaging platform demonstrated proof-of-concept with a robust three-dimensional context to enhance the TEE image guidance for beating heart MV repair with the NeoChord DS1000 device. In this real-time environment, the surgeon easily and intuitively identified the surgical tool, relevant intracardiac anatomy, high-risk areas, and tool trajectories and orientations. Our results demonstrated that tool guidance from the LV apex to the MV leaflet target was always successful with AR guidance, and more importantly, tool navigation was much more accurate (more than threefold to fourfold) with significantly reduced total navigation times (more than fivefold shorter). We also demonstrated a dramatic reduction in the incidence of tool misguidance (and potential injury) into adjacent cardiac structures, such as the aortic valve or anterior leaflet of the MV, thus enhancing patient safety with AR imaging. For example, with TEE guidance alone, the aortic valve was crossed with the tool in nearly all trials, whereas with AR guidance, the aortic valve was never touched at all. These studies also demonstrated the relatively poor tracking ability of two-dimensional echocardiography. Isolated three-dimensional TEE can be used for tracking tools in small spaces; however, it has a significant real-time delay and is often limited in the volume that it can display. As a result, three-dimensional TEE does not easily allow tracking of tools over long distances greater than 5 to 6 cm and cannot predict the optimal tool path for instrument navigation.
When we examined the results by level of expertise, unfortunately, we had too small of a sample size to demonstrate any significant differences. Nonetheless, the confidence intervals of the tool accuracy measurements (distances from intended midline trajectory) were much narrower with AR imaging guidance than TEE alone, suggesting that AR imaging guidance may help improve the reproducibility of results in novice surgeons and shorten the procedural learning curve. In addition, we believe that the AR imaging platform can still provide beneficial guidance to experts as well. Figure 5 demonstrates the superior tool navigation paths with AR imaging compared with TEE alone performed by a surgeon experienced in numerous human implants with the NeoChord DS1000 device. Overall, we believe that AR imaging guidance will shorten procedure times and significantly improve the safety of beating heart MV repair with the NeoChord DS1000 device in both novices and experts. In addition, we believe that the improved image guidance provided by this AR navigation system could be easily adapted to other beating heart procedures. Beating heart MV repair procedures, transcatheter aortic valve implantation, and atrial and ventricular septal defect device closure all rely on limited intraprocedural visualization that could be enhanced with this AR guidance system. Minimally invasive on-pump procedures, such as minithoracotomy mitral and aortic valve procedures, may also benefit from this assisted guidance to accurately place peripheral inserted cannulas centrally.
The infrastructure costs of most currently available intraoperative image guidance systems, including Dyna computed tomography, magnetic resonance imaging, and hybrid operating rooms, are in excess of several millions of dollars.21 Our AR imaging guidance platform is much more cost-effective, with the magnetic tracking system retailing for approximately $18,000 USD and $250 USD for each single-use sensor. In addition, the AR system is portable, avoids irradiation, and does not impede normal operating room workflow.
Limitations in our study include the relatively small sample size, the porcine model, and misregistration errors related to aortic valve and MV annular movement. We are currently working on modifications to provide semiautomatic tracking of the aortic valve and MV annuli to further improve tool guidance accuracy. In addition, we are also developing methods for a more lateral attachment of the neochordae to create a more anatomically accurate chordae tendonae.
We demonstrated proof-of-concept validation for AR-enhanced intracardiac TEE navigation for off-pump transapical MV repair with neochordae implantation. Tracked path analysis clearly showed superior tool navigation accuracy, significantly shorter navigation times, and reduced potential for injury with AR navigation. This study further emphasizes the importance of treating the navigation phase of an image-guided procedure separate from the positioning phase. 19
The authors thank John Zentgraf and Arun Saini (Neo-Chord, Inc., Eden Prairie, MN USA) for assistance with tool development.
1. Enriquez-Sarano M, Akins CW, Vahanian A. Mitral regurgitation. Lancet
. 2009; 373: 1382–1394.
2. Enriquez-Sarano M, Sundt TM III. Early surgery is recommended for mitral regurgitation. Circulation
. 2010; 121: 804–811.
3. Adams DH, Rosenhek R, Falk V. Degenerative mitral valve regurgitation: best practice revolution. Eur Heart J
. 2010; 31: 1958–1966.
4. Braunberger E, Deloche A, Berrebi A, et al.. Very long-term results (more than 20 years) of valve repair with Carpentier’s techniques in nonrheumatic mitral valve insufficiency. Circulation
. 2001; 104 (suppl 1): I8–I11.
5. David TE, Omran A, Armstrong S, Sun Z, Ivanov J. Long-term results of mitral valve repair for myxomatous disease with and without chordal replacement with expanded polytetrafluoroethylene sutures. J Thorac Cardiovasc Surg
. 1998; 115: 1279–1285.
6. Falk V, Seeburger J, Czesla M, et al.. How does the use of polytetrafluoroethylene neochordae for posterior mitral valve prolapse (loop technique) compare with leaflet resection? A prospective randomized trial. J Thorac Cardiovasc Surg
. 2008; 136: 1205.
7. Seeburger J, Falk V, Borger MA, et al.. Chordae replacement versus resection for repair of isolated posterior mitral leaflet prolapse: á ègalité. Ann Thorac Surg
. 2009; 87: 1715–1720.
8. Padala M, Powell SN, Croft LR, Thourani VH, Yoganathan AP, Adams DH. Mitral valve hemodynamics after repair of acute posterior leaflet prolapse: quadrangular resection versus triangular resection versus neochordoplasty. J Thorac Cardiovasc Surg
. 2009; 138: 309–315.
9. Feldman T, Wasserman HS, Herrmann HC, et al.. Percutaneous mitral valve repair using the edge-to-edge technique: six-month results of the EVEREST Phase I Clinical Trial. J Am Coll Cardiol
. 2005; 46: 2134–2140.
10. Feldman T, Kar S, Rinaldi M, et al.. Percutaneous mitral repair with the MitraClip system: safety and midterm durability in the initial EVEREST (Endovascular Valve Edge-to-Edge REpair Study) cohort. J Am Coll Cardiol
. 2009; 54: 686–694.
11. Feldman T, Foster E, Glower DD, et al.. Percutaneous repair or surgery for mitral regurgitation. N Engl J Med
. 2011; 364: 1395–1406.
12. Webb JG, Harnek J, Munt BI, et al.. Percutaneous transvenous mitral annuloplasty: initial human experience with device implantation in the coronary sinus. Circulation
. 2006; 113: 851–855.
13. Harnek J, Webb JG, Kuck KH, et al.. Transcatheter implantation of the MONARC coronary sinus device for mitral regurgitation: 1-year results from the EVOLUTION phase I study (Clinical Evaluation of the Edwards Lifesciences Percutaneous Mitral Annuloplasty System for the Treatment of Mitral Regurgitation). JACC Cardiovasc Interv
. 2011; 4: 115–122.
14. Maisano F, Vanermen H, Seeburger J, et al.. Direct access transcatheter mitral annuloplasty with a sutureless and adjustable device: preclinical experience. Eur J Cardiothorac Surg
. 2012; 42: 524–529.
15. Maisano F, Cioni M, Seeburger J, et al.. Beating-heart implantation of adjustable length mitral valve chordae: acute and chronic experience in an animal model. Eur J Cardiothorac Surg
. 2011; 40: 840–847.
16. Bajona P, Katz WE, Daly RC, Zehr KJ, Speziali G. Beating-heart, off pump mitral valve repair by implantation of artificial chordae tendineae: An acute in vivo animal study. J Thorac Cardiovasc Surg
. 2009; 137: 188–193.
17. Seeburger J, Borger MA, Tschernich H, et al.. Transapical beating heart mitral valve repair. Circ Cardiovasc Interv
. 2010; 3: 611–612.
18. Seeburger J, Leontjev S, Neumuth M, et al.. Trans-apical beating-heart implantation of neochordae to mitral valve leaflets: results of an acute animal study. Eur J Cardiothorac Surg
. 2011; 41: 173–176.
19. Linte CA, Moore J, Wedlake C, et al.. Inside the beating heart: an in vivo feasibility study on fusing pre and intra-operative imaging for minimally invasive therapy. Int J Comp Assist Radiol Surg
. 2009; 4: 113–123.
20. Merk DR, Karar ME, Chalopin C, et al.. Image-guided transapical aortic valve implantation: sensorless tracking of stenotic valve landmarks in live fluoroscopic images. Innovations
. 2011; 6: 231–236.
21. Lang P, Peters TM, Kiaii B, Chu MW. The critical role of imaging navigation and guidance in transcatheter aortic valve implantation. J Thorac Cardiovasc Surg
. 2012; 143: 1241–1243.
22. Vasilyev NV, Novotny PM, Martinez JF, et al.. Stereoscopic vision display technology in real-time three-dimensional echocardiography-guided intracardiac beating-heart surgery. J Thorac Cardiovasc Surg
. 2008; 135: 1334–1341.
23. Gobbi D, Comeau R, Peters T. Ultrasound probe tracking for real-time ultrasound/MRI overlay and visualization of brain shift. Med Image Comput Assist Interv
. 1999; 1679: 920–927.
This is a well-designed experimental study using a porcine model evaluating augmented reality imaging guidance to perform transapical mitral valve repair and chordal reconstruction. The augmented reality system was able to significantly decrease spatial errors, navigation times, and total path lengths compared with echocardiography alone and resulted in a significantly lower potential for injury to other intracardiac structures.
This is an extremely well-performed study and clearly demonstrates the potential impact of introducing more sophisticated imaging guidance systems to facilitate the performance of both transapical and transcatheter procedures. This type of technology has the potential to facilitate and improve the safety of not only beating heart mitral valve repair but potentially other procedures. The limitations of the study include the small sample size and registration errors related to annular aortic and mitral valve annular movement. However, this is a proof-of-concept validation of the potential advantages of augmented reality navigation systems.
Augmented reality image guidance; Mitral valve repair; Chordal reconstruction; Off-pump beating heart surgery; Echocardiography
Copyright © 2012 by the International Society for Minimally Invasive Cardiothoracic Surgery. Unauthorized reproduction of this article is prohibited.
What does "Remember me" mean?
By checking this box, you'll stay logged in until you logout. You'll get easier access to your articles, collections,
media, and all your other content, even if you close your browser or shut down your
To protect your most sensitive data and activities (like changing your password),
we'll ask you to re-enter your password when you access these services.
What if I'm on a computer that I share with others?
If you're using a public computer or you share this computer with others, we recommend
that you uncheck the "Remember me" box.
Highlight selected keywords in the article text.
Data is temporarily unavailable. Please try again soon.