Photogrammetry refers to methods of image measurement and interpretation used to derive an object's shape and location from one or more photographs. Its primary application is three-dimensional (3D) reconstruction of an object in a digital or graphical form.1 Surgeons in facial plastic,2-4 orthopaedic,5 and maxillofacial surgery6,7 have used photogrammetry in morphometric analysis, surgical planning, and outcomes research. This work applies photogrammetry to a novel surgical domain: intraoperative photography.
Technology for creating 3D models is widely available. Professional photograph studios use rigs with multiple aligned cameras to simultaneously capture images for integration. Commercially available software and apps enable individuals to create 3D models using photographs captured on a digital camera or camera phone. Digital spaces exist for sharing, downloading, and printing 3D content. In this work, we apply existing technologies to do circumferential photography in the surgical room and create high-resolution 3D anatomic models. This technique is demonstrated at six time points in an open reduction and internal fixation (ORIF) of the elbow, highlighting key surgical steps and anatomical findings.
Informed consent from human subjects must be obtained and documented. In consultation with the Privacy Office at the University of California, San Francisco, this work was conducted using the institutional consent form for intraoperative photography and external publication.
The photographer and surgeon should confirm key steps of the procedure and agree on a plan for image capture. If retraction is necessary intraoperatively, a set of retractors covered with self-adherent wrap or other nonreflective tape can be prepared in advance.
Ensuring High Quality Images
Reflective objects on the surgical field should be removed or covered with towels or lap sponges. The light reflection from such objects varies between vantage points, changing landmarks used by software algorithms to combine images. This includes metallic surgical instruments, retractors, wires, and pools of liquids.
Surgical lights should be adjusted to illuminate the area of interest. The color temperature of most surgical room lights ranges from 4000 to 4500K and is cooler (more blue) than the flash from most cameras (5000 to 6000K). Photographing with the surgical lights only will ensure the most accurate color rendering of the 3D model.
Objects of interest must remain immobile during photography for good model definition. Any participant in the procedure retracting or holding structures of interest should brace him or herself against stable elements of the surgical field during the session. Photographs should be taken at shorter focal lengths to ensure good field of view. Camera shake or motion artifact can similarly blur the resultant model and should be avoided.
3D models can be produced with photography equipment as basic as a smartphone camera; however, a model produced with a digital single-lens reflex camera or a mirrorless interchangeable lens camera can provide more clarity and detail. Most recent camera models can capture photographs in RAW file formats in which all data from the camera sensor are recorded. Photographs shot in compressed file formats such as JPEG contain only a fraction of the available information. The photographs used in this work were captured using a Nikon D3300 digital single-lens reflex (Nikon). Excellent results can be obtained using a lens with a 55-m focal length, aperture at f/16 or higher for greatest depth of field, and ISO at 200 or 400 depending on the intensity of overhead surgical lighting.
The models in this work were produced from overlapping images taken from two vantage points while moving circumferentially around the surgical field (Figure 1). Additional photographs can be taken to expand the future 3D model; however, incorporating more photographs does not always lead to a sharper model. Individual images should overlap by about 70%, corresponding to about 15 to 20° of circular motion between vantage points (Figure 2).
Postprocessing can help produce a clearer model. Photographs in which the structures of interest are out of focus should be excluded because their inclusion will blur the model. Individual photographs with reflective or thin objects (wires and catheters) should also be removed if possible. Photoediting software can be used to color correct, adjust exposure, and soften harsh shadows that may confuse model-generation software. Postprocessing can uncover excellent images from photographs that may initially appear mediocre.
Many commercially available photogrammetry software options exist. The models in this work were produced with Agisoft Photoscan (Agisoft LLC) because of the ease of its interface and its inexpensive educational license. Model generation involves four main steps. Photographs are aligned based on shared points (Figure 3A) and are used to estimate the locations of the cameras during photography. Next, a dense cloud of points is generated based on depth information calculated from the estimated camera positions (Figure 3B). A polygon mesh approximating the object's surface (Figure 3C) is built from the cloud of common points. Finally, texture is applied to the mesh, giving the model a realistic appearance (Figure 3D).8 The model can be navigated in all coordinate planes and used to do distance and volume measurements when calibrated to a scale object.
Model Export and Sharing
Once the model is complete, it can be exported and edited in a number of 3D file formats, including STL, the primary format used in 3D printing. To view models outside of dedicated 3D viewing software, files can be exported as a 3D PDF file and navigated in three dimensions in versions of Adobe Acrobat Reader version 7.0 and higher (Adobe Systems), a widely used program primarily used for viewing e-documents. 3D content is disabled by default in Acrobat Reader and needs to be enabled by going to edit > preferences > 3D and multimedia and checking “enable playing 3D content.” Models are navigated via holding down the left mouse button with the cursor over the model and dragging the mouse. Additional forms of navigation are available by right clicking the model with the mouse and selecting from the “tools” drop down menu.
The depicted patient was a 51-year-old woman with a neglected chronically dislocated left elbow, left lateral condyle fracture, and distal radius fractures after a traumatic injury. At the time of her revision surgery, she was eight weeks' status postclosed reduction and pinning of the distal radius fracture and failed attempt at elbow stabilization at an outside hospital. The patient was taken to the surgical room for a planned ORIF of the elbow with soft-tissue stabilization as needed.
After positioning, time out, induction of anesthesia, and draping, the lateral aspect of the elbow was incised, revealing the underlying fascia and subcutaneous fat (see Supplemental Figure 1, http://links.lww.com/JG9/A97). On deeper dissection, it was observed that the posterior lateral column of the humerus was fractured and displaced, with the lateral ulnar collateral ligament (LUCL) of the elbow attached to the displaced fragment. A small nondisplaced radial head fracture was also observed (see Supplemental Figure 2, http://links.lww.com/JG9/A98). After the joint was opened and irrigated, the elbow remained irreducible, so a separate medial incision was made.
From the medial side, the ulnar nerve was identified and protected (see Supplemental Figure 3, http://links.lww.com/JG9/A99). After removal of a fragment of bone from the joint, the elbow was reduced. To repair the shredded medial collateral ligament of the elbow, a reconstruction was done with allograft. A 4.5-mm graft was secured to the sublime tubercle with a 4.75 PEEK interference screw (Arthrex) and to the isometric point on the medial distal humerus with an Endobutton Fixation Device (Smith & Nephew). This stabilized the elbow medially (see Supplemental Figure 4, http://links.lww.com/JG9/A100). The flexor pronator muscle group was reattached to the medial epicondyle and the ulnar nerve transposed anteriorly.
On the lateral side, headless screws were inserted to reduce and hold the capitellar/lateral condyle fracture (see Supplemental Figure 5, http://links.lww.com/JG9/A101). The posterolateral fragment was reduced and a posterolateral distal humerus plate was placed to hold the fragments in place (see Supplemental Figure 6, http://links.lww.com/JG9/A102). Locking screws were placed distally, and cortical screws were placed proximally.
The LUCL was stretched and redundant, but repairable. The LUCL was imbricated and reinforced back to bone and to the plate because of the comminution at the bony attachment. The elbow was observed to be stable to varus, valgus, and hanging arm test under fluoroscopic examination. Both incisions were closed in layers, and a long arm posterior splint and a sugar-tong splint were applied for postsurgical stabilization.
This work describes a technique for creating interactive 3D models of surgical anatomy from multiple still photographs. This technique was executed in an ORIF of the elbow with minimal interruption to the case. Our method has been readily done by individuals with limited previous experience in photography using equipment and software that retail for under $500.
1. Luhmann T, Robson S, Kyle S, et al.: Close-Range Photogrammetry and 3D Imaging, ed 2. Berlin, Germany, Walter de Gruyter GmbH, 2014.
2. Taylor HO, Morrison CS, Linden O, et al.: Quantitative facial asymmetry: Using three-dimensional photogrammetry to measure baseline facial surface symmetry. J Craniofac Surg 2014;25:124-128.
3. van der Zeeuw F, Murabit A, Volcano J, et al.: A reliable method to measure lip height using photogrammetry in unilateral cleft lip patients. J Craniofac Surg 2015;26:1865-1870.
4. Weissler JM, Stern CS, Schreiber JE, Amirlak B, Tepper OM: The evolution of photography and three-dimensional imaging in plastic surgery. Plast Reconstr Surg 2017;139:761-769.
5. Furlanetto TS, Sedrez JA, Candotti CT, Loss JF: Photogrammetry as a tool for the postural evaluation of the spine: A systematic review. World J Orthop 2016;7:136-148.
6. Freudlsperger C, Steinmacher S, Bachli H, Somlo E, Hoffmann J, Engel M: Metopic synostosis: Measuring intracranial volume change following fronto-orbital advancement using three-dimensional photogrammetry. J Craniomaxillofac Surg 2015;43:593-598.
7. Worasakwutiphong S, Chuang YF, Chang HW, Lin HH, Lin PJ, Lo LJ: Nasal changes after orthognathic surgery for patients with prognathism and Class III malocclusion: Analysis using three-dimensional photogrammetry. J Formos Med Assoc 2015;114:112-123.
8. Agisoft PhotoScan User Manual: Professional Edition, Version 1.4. St. Petersburg, Russia, Agisoft LLC, 2018.