The paper evaluates the application of a mixed reality (MR) headmounted display (HMD) for the visualization of anatomical structures in complex visceral-surgical interventions. A workflow was developed and technical feasibility was evaluated.
Medical images are still not seamlessly integrated into surgical interventions and, thus, remain separated from the surgical procedure.
Surgeons need to cognitively relate 2-dimensional sectional images to the 3-dimensional (3D) during the actual intervention. MR applications simulate 3D images and reduce the offset between working space and visualization allowing for improved spatial-visual approximation of patient and image.
The surgeon's field of vision was superimposed with a 3D-model of the patient's relevant liver structures displayed on a MR-HMD. This set-up was evaluated during open hepatic surgery.
A suitable workflow for segmenting image masks and texture mapping of tumors, hepatic artery, portal vein, and the hepatic veins was developed. The 3D model was positioned above the surgical site. Anatomical reassurance was possible simply by looking up. Positioning in the room was stable without drift and minimal jittering. Users reported satisfactory comfort wearing the device without significant impairment of movement.
MR technology has a high potential to improve the surgeon's action and perception in open visceral surgery by displaying 3D anatomical models close to the surgical site. Superimposing anatomical structures directly onto the organs within the surgical site remains challenging, as the abdominal organs undergo major deformations due to manipulation, respiratory motion, and the interaction with the surgical instruments during the intervention. A further application scenario would be intraoperative ultrasound examination displaying the image directly next to the transducer. Displays and sensor-technologies as well as biomechanical modeling and object-recognition algorithms will facilitate the application of MR-HMD in surgery in the near future.
Supplemental Digital Content is available in the text
*Charité – Universitätsmedizin Berlin, Berlin, Germany
†Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany
‡Berlin Institute of Health, Department of Surgery, Campus Charité Mitte and Campus Virchow-Klinikum, Experimental Surgery and Regenerative Medicine, Berlin, Germany
§Image Knowledge Gestaltung, Berlin, Germany.
Reprints: Igor M. Sauer, MD, Charité – Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin and Berlin Institute of Health, Department of Surgery, Campus Charité Mitte and Campus Virchow-Klinikum, Experimental Surgery and Regenerative Medicine, Augustenbruger Platz 1, 13353 Berlin, Germany. E-mail: email@example.com.
Both R. Lohmann and J. Pratschke contributed equally.
Funding for the project was solely provided by the Charité – Universitätsmedizin Berlin.
The authors have no conflicts of interest or financial ties to disclose. None of the mentioned companies provided financial, technical, software, or hardware support. Ziosoft Inc., Tokyo, Japan, provided the Ziostation 2 software package and its CT Liver Analysis protocol for evaluation for a period of several months.
The software packages and hardware used for this study are not labeled for the use under discussion and still investigational.
Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (www.annalsofsurgery.com).