Secondary Logo

Journal Logo


Surgical Navigation Technology Based on Augmented Reality and Integrated 3D Intraoperative Imaging

A Spine Cadaveric Feasibility and Accuracy Study

Elmi-Terander, Adrian MD, PhD; Skulason, Halldor MD; Söderman, Michael MD, PhD; Racadio, John MD; Homan, Robert MSc§; Babic, Drazenko MD§; van der Vaart, Nijs PhD§; Nachabe, Rami PhD§

Author Information
doi: 10.1097/BRS.0000000000001830
  • Open


Placement of pedicle screws is challenging and poses risks, as vital portions of the spinal anatomy and neurovascular structures are not visible to the surgeon. Traditionally, pedicle screw placement with free-hand technique relies on anatomical landmarks and preoperative imaging. X-ray fluoroscopy is frequently used to provide guidance and confirm adequate screw placement, despite radiation exposure to the medical staff and patients. Surgical navigation systems coupled with intraoperative volumetric imaging are used for various clinical applications in spine surgery, requiring pedicle screw placement.1 Current systems typically consist of an infrared-camera tracking unit linked to an intraoperative computed tomography (CT) and a reference frame attached to the spinous vertebral process for registration and tracking purposes.2

Commercially available surgical navigation systems have shown superiority in pedicle screw placement when compared with free-hand technique in the lumbosacral region.2 In the thoracic spine, pedicle screw placement accuracy remains a challenge because of the morphology (e.g., smaller pedicle size).3–8 Lateral or medial cortical breaches in thoracic pedicles can potentially yield severe clinical complications due to the proximity of critical structures.9

Recently, cardiothoracic, cerebrovascular, and abdominal vascular surgeries have pioneered device navigation in hybrid operating rooms (H-OR). In an H-OR, an operating table is interconnected to a motorized flat panel detector C-arm with 2D/3D imaging, along with x-ray based image guidance tools.10,11 In this study, we introduce an H-OR surgical navigation platform with optical cameras for augmented reality in intraoperative imaging and tracking. The optical platform was integrated with the standard flat detector frame of a motorized C-arm. We assessed the feasibility and accuracy of pedicle screw placement in the thoracic spine comparing ARSN with free-hand technique.


Study Design

The study was conducted in compliance with the ethical guidelines for human cadaver studies. Body donors allowed their bodies or body parts to be used for research and educational purposes. Two senior consultant neurosurgeons with expertise in cervical and upper thoracic spinal fixation performed all surgical procedures.

Four human cadavers with no history of spine surgery were used. The cadavers were placed prone on the surgical table. Radiological examinations of the spine revealed that two of the cadavers had a rotatory deformity of the upper thorax (up to 15° axial rotation). The other two cadavers had normal spinal anatomy. The whole dorsal thoracic region was exposed after performing a midline incision followed by complete bilateral dissection and retraction of the paraspinal musculature down to the spinous processes and tips of the transverse processes.

Pedicle screws of 3.5 or 4.5 mm in diameter (Medtronic, Minneapolis, MN) were introduced in the thoracic spines with the ARSN technology unilaterally (left or right) and with free-hand technique on the contralateral side. The screws were placed in the cranio-caudal direction three spinal levels at a time, always beginning with free-hand technique by one surgeon and subsequently switching to the contralateral side, where another surgeon used ARSN to position the screws. Each surgeon operated on either side and with either technique.

Augmented Reality Surgical Navigation Technique

The study was conducted in a hybrid room equipped with a Maquet surgical table (Alphamaquet 1150, Maquet AG, Switzerland) connected to a motorized ceiling-mounted C-arm flat detector system, enabling high-resolution cone-beam CT acquisition (AlluraClarity FD20; Philips Healthcare, Best, the Netherlands). The navigation system, developed by Philips Healthcare, consisted of four high resolution (5 megapixels) optical video cameras mounted in the frame of the flat detector and directed toward the isocenter of the C-arm (Figure 1).

Figure 1:
Hybrid operating room with (a) surgical table, (b) ceiling-mounted motorized C-arm with 2D/3D imaging capabilities allowing cleared space for anesthesia at the head side, (c) Medical grade monitor, (d) flat detector with four integrated optical cameras (e) at each side directed toward the isocenter of the C-arm for tracking, navigation, and displaying live images with augmented reality on the medical monitor (zoom-in on one of the cameras on the top left).

Before starting the procedure, 8 to 10 sterile, flat, adhesive circular markers were randomly placed on the skin around the surgical site after skin incision and surgical exposure of the whole thoracic spine. The markers were specifically designed to be tracked by the system. A minimum of five markers had to be in the field of view of the cameras for patient tracking. In order to compensate for any motion, a mesh model generated by interconnecting the markers was used throughout the procedure as a patient tracking model (Figure 2).

Figure 2:
Example of a mesh interconnecting the markers to create a patient model for motion tracking and compensation.

The flat detector could cover multiple vertebrae in one single rotational scan.12 Three spinal levels were imaged by a 3D cone beam CT (CBCT) acquisition. The C-arm rotated 180° in 20 seconds, and within 15 seconds, a 3D reconstructed volume with 0.5 mm voxel size was displayed as 1 mm thickness axial and sagittal slice views with a contrast resolution of 3 to 5 HU. The spine was automatically segmented (i.e., the spine bony contour delineated) and displayed as a 3D volume in addition to two orthogonal projections, in order to enable planning of pedicle screw insertion path. Virtual screw size and placement could be modified on the 3D workstation (Figure 3). All images were displayed on a 58-inch high definition medical monitor.

Figure 3:
Automatic vertebral body (in white color) and pedicles (in blue color) segmentations (left viewport). Virtual screw placement after selection of the pedicle of interest.

The C-arm automatically rotated into one of the four possible positions where an optical camera was aligned along the planned axis of insertion. The other three optical cameras provided angulated views for the alignment of the devices. The screw path was overlaid on the optical camera views depicting the intended device alignment as well as 2D slices or 3D volume rendering from the acquired CBCT (Figure 4). The optical and intraoperative images were automatically registered and therefore the operator did not interact with the system for registration. During screw insertion, the optical overlay provided real-time feedback to the operator. X-ray fluoroscopy was not used.

Figure 4:
Augmented reality with optical camera based navigation and tracking, viewport on the left correspond to the view along the axis of the device, top right and bottom right correspond to oblique view from two other cameras for fine alignment of the device along the virtual path and screw augmented with a CBCT slice. In this particular example, the operator desired to have two alignment views only.

CBCT was performed to evaluate the position of the inserted screws. Angulated scans were used in order to minimize metal artifacts (Figure 5).

Figure 5:
Examples of 3D CBCT acquired postoperatively. Note the complete absence of streaking (i.e., beam hardening) artifact despite the presence of four screws, as the CBCT was performed at +20° angulation of the C-arm.

Free-Hand Surgical Technique

The entry point was identified using posterior anatomical landmarks. An awl was used to perforate the cortical bone. A gearshift probe was subsequently advanced in the cancellous bone to create a path in which the screw was inserted.

Pedicle Screw Accuracy Evaluation

Postoperative CBCT images were transferred to a PACS system without any annotation or indication of screw placement technique (Figure 6). Two independent neurosurgeons and two radiologists assessed screw positions using the Gertzbein grading: grade 0 (screw within the pedicle without cortical breach), grade 1 (0–2 mm breach, minor perforation including cortical encroachment), grade 2 (2–4 mm breach, moderate breach), and grade 3 (more than 4 mm breach, severe displacement).13 The raters could modify the window width and level as well as the 3D volume display and multi-planar reconstruction.

Figure 6:
Example of anonymized oblique axial, sagittal, and coronal slices along pedicle screw axis for grading. Note that the type of surgical technique was blinded to the rater. The ability of visualizing three different slices simultaneously helped to appreciate cortical encroachment.

Morphological Assessment

In order to evaluate potential factors that induced breaching, we measured the axial and sagittal widths and angles of each pedicle, as well as the axial and sagittal rotations of each vertebral body on the planning CBCT (Figure 3).14

Statistical Analysis

Two-way intraclass correlation coefficient for absolute interrater agreement (IA) was calculated with 95% confidence interval (95% CI) to assess the absolute agreement among the four raters in grading the screw position accuracy. An IA below 0.40, between 0.40 and 0.59, between 0.60 and 0.74, and above 0.75 corresponded to poor, fair, good, and excellent agreement, respectively.15 Continuous variables were expressed as means and standard deviations, whereas ordinal and categorical variables as frequencies, ratios, or percentages.

The median value of the grades for each pedicle was considered as the consensus grade for further data analysis. One-tailed Wilcoxon signed rank test was used to compare accuracy for both groups, and Fisher's exact test was used for nominal variables comparison. Relative risks (RRs) along with 95% CI were calculated. We used one-tailed ordinal Spearman's correlation (ρ) to identify which of the morphological factors correlated with breaching. Statistical analysis was performed using Matlab R2015a (Mathworks, Natick, MA). A P value (P) less than 0.05 was considered as significant.


The IA was 0.78 (95% CI: 0.70–0.84) indicating excellent IA. Grades 0 screw placement was roughly 71% more frequent with ARSN (P < 0.05), grade 1 and 2 were statistically similar for both methods, and a significantly (P < 0.05) lower amount of grade 3 was achieved with ARSN (Table 1). Accuracy was significantly higher in the ARSN group (85%, 40/47) than the free-hand group (64%, 30/47) yielding an RR = 1.33 (95% CI: 1.04–1.71). ARSN demonstrated higher accuracy than free-hand technique analyzed pairwise for each level (P = 0.0008). Twenty-three vertebral bodies showed larger breaches with free-hand technique than ARSN (Table 2). For both techniques, two screws were displaced medially; at T5 and T12 with ARSN, T4 and T11 with free-hand. Table 3 summarizes the spinal levels were cortical encroachment (grade 1) and displaced screws (grades 2 and 3) were observed along with direction of displacement.

Screw Positioning Accuracy Comparison Between the ARSN and Free-Hand Groups Using Gertzbein Grading System
Contingency Table Indicating the Corresponding Grade of the Screws Placed With the ARSN on One Side of the Spine and Free-Hand on the Opposite Side to Match for Each Vertebral Body Complexity
Breaching Per Spinal Level and Corresponding Direction

The correlation factors between the morphological dimensions of each pedicle and the grade of screw placement are summarized in Table 4. The accuracy of screw placement with ARSN did not show statistical correlation with any of the morphometrical factors except for the pedicle axial width suggesting higher breaching for smaller pedicle axial width. Screws placed with the free-hand technique negatively correlated with smaller pedicle widths (i.e., decreasing grade values with increasing width) and positively correlated with larger pedicle angles. Vertebral body sagittal rotation demonstrated the highest correlation (ρ = 0.51).

Correlation Factors ρ Between Grades and Various Morphometrical Dimensions With 95% Confidence Intervals (CI) and P Value


This study proved that accurate spinal pedicle screw placement was feasible using augmented reality surgical navigation technology. In the thoracic spine, the achieved accuracy (85%) with ARSN was significantly (P < 0.05) better than screw placement with free-hand technique.

The ARSN system comprises navigation technology integrated in the flat detector frame, eliminating the need for stand-alone imaging and navigation equipment.2 Several levels can be planned with the same degree of accuracy, without the need for invasive dynamic reference frame positioning and recalibrations. Several other studies have highlighted technical reasons for inaccurate screw placement. For example, screw misplacement correlated with the increased distance from the treated vertebral body to the reference frame, infrared cameras obstructed the line of sight, calibration errors occurred, and inadvertently touching the reference frame caused registration error.6,7,16 Besides the advantage of being noninvasive, the ARSN technology utilizes a patient tracking algorithm that identifies the location of flat adhesive markers placed on the skin. The markers are tracked, as long as at least five markers are still visible from any of the four cameras, thus minimizing the risk for obstruction of the line of sight between the markers and the cameras.

The accuracy between navigation systems and free-hand technique for thoracic screw placement has been compared in other studies.5,6,17–21 In this context, it should be noted that in our study, the screw placement grading was more rigorous than in comparable studies. We used three different views (axial, coronal, and sagittal) to identify any potential breach in any direction and were thus more likely to identify a subtle breach not visible on axial views only (cf. Figure 6). Despite this, our results with both techniques were comparable to published data (cf. Table 5). In one case, we observed a grade 3 breach when using ARSN, whereas the contralateral screw was grade 0. In this case, it was impossible to properly access the entry point due to the local anatomy (a cortical ridge displaced the screw).

Literature Sorted in Chronological Order Where Navigation With 3D Intraoperative CT Was Compared With a Conventional Technique, and Their Main Findings

Few studies have evaluated IA with values ranging from 0.45 to 0.85, with a single study showing excellent agreement (i.e., above 0.75).22 Our study demonstrated an IA of 0.78 (excellent agreement), which is paramount for the reliability of the grading.

The accuracy values in literature summarized in Table 5 correspond to large cohorts and consequently yield more experience with the used technology, whereas the surgeons in this study did not have prior training in ARSN. The learning process of current navigation systems can be slow when transitioning from conventional surgical methods.23,24 Rivkin and Yocom23 evaluated the accuracy after every 15 consecutive patients undergoing spinal surgeries in the thoracolumbar region and showed a linear increase in accuracy from 82.9% after 15 cases to 98.9% after 270 cases, using a Stealth Station navigation system with O-arm CT imaging (Medtronic, Minneapolis, MN). Ryang et al.24 concluded the same by looking at each quarter (average of 40 screws per quarter) for a year time span and found an increase in accuracy in thoracic spine starting from 55% after 40 screws up to 87.5% after a total of 192 screws placed, using the Vector Vision navigation system (Brainlab, Munich, Germany) with Siemens Orcadis C-arm with CBCT intraoperative imaging. In our study, we found an 85% accuracy after only 47 screws insertion (22 and 25 per surgeon). The Gertzbein grading is a tool for accuracy and not a surrogate for clinical outcome. In fact, Gautshi et al.9 showed in a literature review that no complications (dural lesions, nerve root injuries, etc.) were observed in studies where accuracy was as low as 78%. Therefore, care should be taken not to correlate clinical impact and accuracy grading.

Several studies have highlighted the anatomical challenges for the upper thoracic (i.e., Th1–6) parts where pedicles can be in average as small as 4 mm in width, and there is a wide variety of interpatient pedicle angle/axis.14 Most studies evaluated the correlation between accuracy and the spinal level; however, we decided to assess the correlation between accuracy and each anatomical dimension, as it seems to give a better understanding of the risk factors for breaching; each spinal level can have a wide interpatient variability.5,8,14 In the free-hand group, morphometrics of the pedicles and the spinal curvatures were risk factors (cf. Table 4), whereas in the ARSN group, the pedicle axial width was the only risk factor. The 3D cone beam CT provides spatial information that allows to cope with the various angulations of the pedicles and spine allowing a more accurate surgical tool placement during the procedural phase of screw insertion. The pedicle axial width is the only risk factor for breaching that is explained by the fact that the smaller the pedicle width, the higher the odd to breach, especially when using a screw size (e.g., 3.5 mm in diameter) that barely fits in the pedicle (e.g., 3 mm in diameter) causing cortical encroachment.

ARSN does not rely on perioperative x-ray fluoroscopy but on optical tracking. As there is no radiation during the screw placement, there is no need for protective lead aprons during surgery. Only one single cone beam CT is needed to perform the planning of the screw placement using ARSN that would be the only source of patient dose. However, one can take the advantage of being able to perform a postoperative cone beam CT in an H-OR before closing the wounds to revise potential malpositioned screws in order to avoid reoperations.

Limitations of our study include the small sample size and procedural time. Further investigation is required to quantify the learning curve effect. Another limitation is that the study was performed on cadavers and not patients with present spinal pathologies. The effect of patient ventilation on patient tracking accuracy has to be assessed. Other studies showed reduction of screw placement accuracy from respiratory movement.16 Radiation dose to cadavers from the 3D cone beam CT acquisition was not evaluated, which is another limitation that needs to be addressed in future studies.


H-OR with integrated optical-based augmented reality navigation on a C-arm imaging system allowed thoracic screw placement with an accuracy of 85% for thoracic pedicle screw placement without prior training of the operating surgeons. Matched for complexity level, the accuracy was 33% higher than free-hand technique (85% vs. 64%, P < 0.05). There was no periprocedural fluoroscopy.

Key Points

  • Thoracic pedicle screw placement using an optical-based navigation technology with augmented reality and integrated intraoperative imaging is feasible.
  • Patient tracking for spinal navigation with noninvasive, flat adhesive markers placed on the skin is feasible.
  • The overall accuracy of pedicle screw placement with the ARSN system was significantly better than with the free-hand surgical technique (85% vs. 64%, P < 0.05).
  • The ARSN system eliminates the need for periprocedural x-ray fluoroscopy.


The authors thank Drs. Neil Johnson, Kyrre Pedersen, and Eric Edstrom for rating the pedicle screw placements. The authors would also like to thank Nicole Hilvert for her assistance during the experiments.


1. Tian NF, Huang QS, Zhou P, et al. Pedicle screw insertion accuracy with different assisted methods: a systematic review and meta-analysis of comparative studies. Eur Spine J 2011; 20:846–859.
2. Helm P, Teichman R, Hartmann S, et al. Spinal navigation and imaging: history, trends, and future. IEEE Trans Med Imaging 2015; 34:1738–1746.
3. Kosmopulos V, Schizas C. Pedicle screw placement accuracy: a meta-analysis. Spine 2007; 32:111–120.
4. Shin BJ, James AR, Njoku IU, et al. Pedicle screw navigation: a systematic review and meta-analysis of perforation risk for computer-navigated versus freehand insertion. J Neurosurg Spine 2012; 17:113–122.
5. Waschke A, Walter J, Duenisch P, et al. CT-navigation versus fluoroscopy-guided placement of pedicle screws at the thoracolumbar spine: single center experience of 4500 screws. Eur Spine J 2013; 22:654–660.
6. Kraus M, Weiskopf J, Dreyhaupt J, et al. Computer-aided surgery does not increase the accuracy of dorsal pedicle screw placement in the thoracic and lumbar spine: a retrospective analysis of 2003 pedicle screws in a level I trauma center. Global Spine J 2015; 5:93–101.
7. Tabaraee E, Gibson AG, Karahalios DG, et al. Intraoperative cone beam computed tomography with navigation (O-arm) versus conventional fluoroscopy (C-arm). A cadaveric study comparing accuracy, efficiency, and safety for spinal instrumentation. Spine 2013; 38:1953–1958.
8. Akazawa T, Kotani T, Sakuma T, et al. Evaluation of pedicle screw placement by pedicle channel grade in adolescent idiopathic scoliosis: should we challenge narrow pedicles? J Orthop Sci 2015; 20:818–822.
9. Gautshi OP, Schalto B, Shaller K, et al. Clinically relevant complications related to pedicle screw placement in thoracolumbar surgery and their management: a literature review of 35630 pedicle screws. Neurosurg focus 2011; 31:1–9.
10. De Ruiter QM, Moll FL, Gijsberts CM, et al. AlluraClarity radiation dose reduction technology in the hybrid operating room during endovascular aneurysm repair. J Endovasc Ther 2016; 23:130–138.
11. Fandino J, Taussky P, Marbacher S, et al. The concept of a hybrid operating room: applications in cerebrovascular surgery. Acta Neurochir Suppl 2013; 115:113–117.
12. Braak SJ, Zuurmond K, Aerts HC, et al. Feasibility study of needle placement in percutaneous vertebroplasty: cone-beam computed tomography guidance versus conventional fluoroscopy. Cadiovasc Intervent Radiol 2013; 36:1120–1126.
13. Gertzbein SD, Robbins SE. Accuracy of pedicular screw placement in vivo. Spine 1990; 15:11–14.
14. Zindrick MR, Wiltse LL, Doornik, et al. Analysis of the morphometric characteristics of the thoracic and lumbar pedicles. Spine 1987; 12:160–166.
15. Schrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979; 86:150–155.
16. Scheufler KM, Franke J, Eckardt A, et al. Accuracy of image-guided pedicle screw placement using intraoperative computed tomography-based navigation with automated referencing. Part II: thoracolumbar spine. Neurosurgery 2011; 69:1307–1317.
17. Lekovic G, Potts E, Karahalios, et al. A comparison of two techniques in image-guided thoracic pedicle screw placement: a retrospective study of 37 patients and 277 pedicle screws. J Neurosurg Spine 2007; 7:393–398.
18. Sakai Y, Matsuyama Y, Nakamura H, et al. Segmental pedicle screwing for idiopathic scoliosis using computer-assisted surgery. J Spinal Disorder Tech 2008; 21:181–186.
19. Cui G, Wang Y, Kao TH, et al. Application of intraoperative computed tomography with or without navigation system in surgical correction of spinal deformity. Spine 2012; 37:891–900.
20. Allam Y, Silbermann J, Riese F, et al. Computer tomography assessment of pedicle screw placement in thoracic spine: comparison between free hand and a generic 3D-based navigation techniques. Eur Spine J 2013; 22:648–653.
21. Jin M, Liu Z, Liu X, et al. Does intraoperative navigation improve the accuracy of pedicle screw placement in the apical region of dystrophic scoliosis secondary to neurofibromatosis type I: comparison between O-arm navigation and free hand-technique. Eur Spine J 2016; 25:1729–1737.
22. Aoude AA, Fortin M, Figueiredo R, et al. Methods to determine pedicle screw placement accuracy in spine surgery: a systematic review. Eur Spine J 2015; 24:990–1004.
23. Rivkin MA, Yocom SS. Thoracolumbar instrumentation with CT-guided navigation (O-arm) in 270 consecutive patients: accuracy rates and lessons learned. Neurosurg Focus 2014; 36:E7.
24. Ryang YM, Villard J, Obermuller T, et al. Learning curve of 3D fluoroscopy image-guided pedicle screw placement in the thoracolumbar spine. Spine J 2015; 15:467–476.

accuracy; augmented reality; hybrid operating room; integrated navigation; occupational radiation dose; pedicle screw

Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.