With the rapid development of three-dimensional (3D) reconstruction technology, it is becoming more widely used in the clinical setting. In addition to being applied to bones, the technology can also be applied to organs with rich vascular systems, such as the heart, liver, and maxillofacial area.[1–3] Due to the demands of modern surgery, 2D images are no longer enough and 3D images provide additional visualization information that is crucial in surgical planning (Fig. 1). Using 3D reconstruction technology, the area of focus and its surrounding tissues and blood vessels can be accurately located. This enables us to understand the relevant anatomical variations and realize accurate preoperative and individualized operation scheme formulation and more accurate intraoperative positioning given the technology’s 3D and intuitive advantages (Fig. 2). It is even used in the prediction of postoperative complications and surgical teaching. Despite the continuous progress of the clinical treatment methods of pancreatic surgery, the anatomical relationship of peripancreatic organs is complex and changeable, the risk of bleeding in pancreatic surgery is high, there are many postoperative complications, the prognosis is poor, and there is a low 5-year survival rate. As pancreatic surgery is still a difficult problem for surgeons, the clinical application of 3D reconstruction technology in the diagnosis and treatment of pancreatic surgery has broad prospects that are worthy of in-depth study.
Database search strategy
We conducted a systematic literature search using the PubMed database. The following keywords were selected: pancreatic surgery, three-dimensional reconstruction, vascular reconstruction, three-dimensional visualization, augmented reality (AR), virtual reality, surgery, and clinical application. We selected only English language and full-text articles, and most of the selected studies (more than 75% of the overall references) were published between 2015 and 2022.
Application of 3D reconstruction in the preoperative scheme of pancreatic surgery
Whether pancreatic surgery can preserve normal pancreatic parenchyma as much as possible and reduce surgical bleeding under the condition of R0 resection depends on the judgment of preoperative imaging such as the lesion size and vascular invasion. The arteries involved mainly include the celiac trunk, superior mesenteric artery (SMA), common hepatic artery, gastroduodenal artery, and the first jejunal branch. The veins mainly include the superior mesenteric vein (SMV), portal vein, inferior vena cava, splenic vein, and proximal jejunal drainage branch. Marconi et al. proposed a 3D reconstruction tool for pancreatic ductal adenocarcinoma that uses images collected by multidetector Computed Tomography (MDCT) to semiautomatically establish a 3D virtual model. The final 3D virtual model included the peripheral organs of the pancreas and the main blood vessels around the pancreas, including the aorta and main peripancreatic arteries, portal vein and its main branches, inferior vena cava, gallbladder, and biliary ducts, Wirsung duct, second and third parts of the duodenum, and biliary stent, if present. This 3D virtual model enables the surgeon to more directly and grasp the anatomical information of the patient’s peripancreatic vessels.
The normal anatomical structure of the common hepatic artery is derived from the celiac trunk. Sitarz et al. used maximum intensity projection and 3D volume-rendering techniques to noninvasively display the arteriole in MDCT. A preoperative understanding of the abnormal common hepatic artery was derived from the SMA in the patient. Despite the anomalous origin of the CHA, pylorus-preserving pancreatoduodenectomy (PD) and regional lymph node dissection without intraoperative complications were performed in each case. Understanding the anatomical structure of the inferior pancreaticoduodenal artery (IPDA) is of great significance for patients undergoing PD, especially in the artery-first approach. Timaru et al. conducted 3D reconstruction of MDCT of 714 patients and found 139 cases of the ectopic right hepatic artery, among which 69 cases of IPDA were derived from SMA. The anterior and posterior branches of IPDA were derived from SMA and the ectopic right hepatic artery in 23 cases, respectively, and IPDA was derived from the ectopic right hepatic artery in 11 cases. The latter 2 variants undoubtedly cause more difficulties for the preservation of the ectopic right hepatic artery in PD, and preoperative 3D reconstruction can enable surgeons to consider treatment methods in advance.
Miyamoto et al. combined MDCT and magnetic resonance cholangiopancreatography (MRCP) images and applied them to the 3D surgical simulation of pancreatic surgery using the SYNAPSE VINCENT medical imaging system, and preoperative understanding of the hepatic artery, inferior mesenteric vein (IMV), and left gastric vein (LGV) could reduce surgical blood loss. It could also avoid injury to right hepatic artery originating from SMA and the left hepatic artery originating from the left gastric artery, as well as protect the blood supply of the liver. The technology also allows the operation personnel involved with the preoperative shared pancreas 3D reconstruction image to formulate a relevant operation scheme. A randomized trial showed that preoperative sharing of 3D reconstruction images of the pancreas could help reduce intraoperative bleeding compared with not being able to become familiar with the organs and vascular structures of the surgical area with 3D reconstruction images.
Adequate understanding of the anatomy of the proximal jejunal and pancreatic veins is a prerequisite for safe PD surgery, but there is scarce literature detailing the description of the first and second jejunal veins (J1V, J2V). Ishikawa et al. performed MDCT scanning and venography in 155 patients. The SYNAPSE 3D system was used to generate 3D-CT images detailing the anatomy of the jejunal vein. J1V and J2V formed the first jejunal trunk in 84% of patients, usually located ventrally in SMA, and it was mainly drained into the SMV, with a small amount of drainage into the splenic portal vein confluence. Three-dimensional reconstruction images can provide anatomical theoretical support for mastering the movement of these vessels, but whether they can reduce intraoperative bleeding and postoperative complications remains to be studied.
Tumor diameter and location of tumor-associated vessels are risk factors for laparoscopic tail pancreatectomy.[14,15] In addition to clearly showing the surrounding structures before surgery, 3D reconstruction can predict the risk of failure of spleen-preserving laparoscopic distal pancreatectomy (LDP) by calculating the contact area between the resected pancreas and the splenic vein as well as the maximum ratio of the length of the spleen vessels buried in the resected pancreas to the length of the spleen vessels. Zhou et al. divided the patients into the splenic vascular-sparing LDP group and the nonsplenic vascular-sparing LDP group according to the surgical method. The clinical data of the 2 groups were compared to evaluate the risk factors for splenic vessel preservation failure. Multivariate analysis confirmed that large tumor size, a large contact area between the pancreas and splenic vein to be resected, and a large maximum ratio of the splenic vessel length to the splenic vessel length in the pancreas to be resected were all independent risk factors for splenic vessel preservation failure.
Application of 3D reconstruction in pancreatic surgery
Owing to the aggressiveness of pancreatic cancer and the close relationship between the tumor and the location of blood vessels, pancreatic surgery has always been a difficult type of abdominal surgery. Preoperative two-dimensional imaging examination makes it difficult to determine the anatomical relationship between the pancreatic tumor and peripheral blood vessels or vascular route and tumor invasion of blood vessels, making it easy to cause accidental injury to blood vessels during surgery. Three-dimensional reconstruction is more consistent with human visual habits and can be used to comprehensively observe the anatomical relationship between lesions and surrounding organs and vessels, completing the resectable evaluation. It also enables surgeons to understand the possible obstacles in the process of surgery, which is beneficial to protect blood vessels during surgery, reduce the operation time, and help surgeons choose the best surgical approach.
Fang et al. conducted a randomized trial to confirm the feasibility of peripancreatic organ reconstruction in patients with pancreatic cancer. MDCT images and data of patients were segmented and reconstructed automatically using the Medical image 3D visualization system (MI-3DVS) to segment important structures such as the liver, pancreas, pancreatic tumors, peripancreatic vessels, spleen, splenic arteries, and veins. The results show that there was no significant difference in age and tumor size between the 2 groups. The group that could observe the 3D reconstruction model before and during surgery had significantly reduced blood loss and shortened operation times.
The AR-based navigation system (NS) in surgery can superimpose a 3D reconstruction model of preoperative image generation on a real view of the surgical area to provide the surgeon with visual anatomy, to help them identify organs and blood vessels throughout the surgery. Onda et al. used the imaging software Analyze to manually or semiautomatically delineate the surrounding structure of the pancreas in the MDCT images of patients before surgery and give colors to reconstruct the 3D image, which was applied to the NS based on AR. With this technique, surgeons can observe vascular abnormalities during surgery. Visual observation of the vascular position, angle, and direction according to the preoperative reconstruction image on the display is beneficial for the early identification of IPDA or jejunal artery. However, this method requires expensive stereoscopes, and surgeons have to exclude their vision from the field of surgery, affecting the continuity of surgery and increasing the operation time. Therefore, the team developed a way to use a tablet computer to view 3D images during surgery. The 3D organ model can be superimposed on a tablet computer monitor. A tablet computer monitor captures images of the surgical area in real-time, and the operator can choose any organ on the touch screen. When viewed from any angle, the overlapped image responds well and follows the motion of the tablet, facilitating an intuitive understanding of the anatomy. This method is not only low cost but also allows surgeons to view overlapping images from any angle during surgery without switching between the surgical field and the screen. It is beneficial to the continuity of operation and shortens the operation time.
However, AR-based NS has many problems to overcome, such as the deformity of organ reconstruction, low practicality for surgery, and high cost. Therefore, Okamoto et al. focused on the development of AR-based NS for pancreatic surgery. Because pancreatic surgery has minimal displacement compared to gastrointestinal surgery, the position of each organ in the overlapping image almost corresponds to the actual organ, with an average registration error of approximately 5 mm. This provides an idea for solving the malformation of reconstructed organs.
Application of 3D reconstruction after pancreatic surgery
Cinematic rendering (CR) is a photorealistic visualization method for volumetric imaging data that are being investigated for a variety of potential applications, including the realistic display of complex anatomical structures and the visual characterization of mass lesions. Pancreatic cancer is highly invasive, and vascular invasion occurs in some surgical patients; this complication inevitably requires intraoperative vascular reconstruction, which is associated with the risk of surgical bleeding. Recently, solid shadow rendering has been used to evaluate complications associated with vascular resection and reconstruction, such as bleeding, pseudoaneurysm, vascular thrombosis, and ischemia.[24,25] The application of CR in postoperative image reconstruction of pancreatic surgery is conducive to the accurate judgment of postoperative complications and provides an image basis for early treatment of complications.
With technological advances in tissue clearance, antibody penetration, and advanced microscopy, 3D visualization of dense fibrous tissue is now possible. It is conducive to more accurate detection of vein invasion in postoperative pathological specimens. Venous invasion, which is prevalent in pancreatic cancer, has been detected more frequently in 3D assessments than in traditional flat slide assessments. This suggests that even surgically resectable PDAC can invade the vein and gain the ability to spread widely. Therefore, 3D visualization of dense fibrous tissue provides a more accurate detection tool for postoperative pathological analysis of pancreatic cancer.
Application of 3D reconstruction in pancreatic surgery teaching
With the rapid development of digital technology, 3D anatomical reconstruction has gained more potential value in surgical medical education. Unlike the previous simple textbook language description or 2D image description, 3D reconstruction can more clearly show the adjacent relationship of anatomical results. The use of virtual or AR technology can motivate students. It can shorten the surgical learning curve for young doctors with insufficient clinical experience and insufficient understanding of spatial structure.
Three-dimensional reconstruction can improve surgical residents’ assessment of resectable pancreatic cancer. Bailer et al. used MeVis 3D reconstruction software to transform high-quality CT scans into 3D models. Three-dimensional reconstruction included pancreas and lesions, portal vein vessels, inferior vena cava vessels, and major arteries. The subjects were divided into 2 groups: senior residents who had more clinical experience and junior residents who had just entered the clinic. In terms of objective assessment of resectability, senior residents with more knowledge and surgical experience could benefit from vascular 3D reconstruction by accurately determining the resectability of pancreatic cancer using 3D reconstruction. The junior residents performed worse than the senior residents objectively owing to their lack of clinical experience. On the subjective side, both groups of physicians were more confident in tumor resectability and R0 resection after obtaining the 3D reconstruction model.
Three-dimensional reconstruction helps surgical students deal with more complex clinical problems. Lin et al. developed 3D training of pancreatic cancer blood vessels and peripancreatic tissues by using the Multi-touch Visualization table (MVT) for 3D reconstruction to evaluate its value in anatomy–image–surgery knowledge system construction and surgical planning for training surgical trainees. The results show that surgical trainees trained in 3D reconstruction models performed better on questions related to tumor staging and surgical planning, and participants in the 3D group agreed that 3D was better for understanding and planning pancreatic surgery than those in the 2D group. This may be indirect evidence that surgical residents have insufficient visuospatial ability to accurately interpret 2D images as 3D images. It also demonstrates the advantages of 3D virtual models in simplifying complex structures and improving the understanding of spatial relationships between tumors and adjacent structures.
Limitation and new insights into 3D reconstruction of the pancreas
Currently, most 3D reconstruction of the pancreas is based on 2D images, and the layer spacing of the image has a decisive influence on the quality of the 3D image. Successful 3D reconstruction largely depends on the thickness of the image and the stability of the patient during CT. Manual or semiautomatic 3D reconstruction takes time and energy, and professional physicians need to spend a certain amount of time and energy to identify each tissue in places with poor contrast. If an automated reconstruction tool can be established, 3D reconstruction will be greatly promoted for clinical services. Deep learning (DL) is a branch of artificial intelligence. By using the convolutional neural network, DL can automatically recognize and distinguish different organizations in pictures through the accumulation of training sets. Using the DL method, 3D reconstruction can be completed automatically and become more accurate with the increase in the number of reconstruction cases. This method will have great potential in the clinical setting.[29,30]
Although 3D reconstruction meets the needs of vision, it cannot exist in reality, lacks tactile experience, and cannot be used as a physical object in clinical practice. Three-dimensional printed specimens can simulate the texture and color of real organs, transforming abstract knowledge and complex cases into visible and touchable models. Other studies have applied 3D printing based on 3D reconstruction to clinical applications, such as hearts and livers.[31,32] Pancreatic surgery has also been employed in research on 3D printing. 3D printing allows researchers to observe the relationship between the tumor and the surrounding organs and blood vessels from all aspects of the physical object, reducing the difficulty of surgery and improving the safety and accuracy of surgery. However, its cost is relatively high, and teaching can only be done once. A novel 3D-printed biopolymer device (3DP-BPD) has been used in mini pigs to verify the feasibility and effectiveness of 3DP-BPD in duct-to-mucosa pancreaticojejunostomy. It provides experimental evidence for clinical application. It is believed that with the improvement of technology and materials, 3D printing based on 3D reconstruction can also be widely applied in pancreatic surgery.
Errors between 3D reconstruction and the real surgical field are unavoidable. If an appropriate imaging medium can be found and used together with 3D reconstruction in pancreatic surgery to provide image support for tumor localization and vascular development, it can enable safe operations involving pancreatic cancer and promote the development of precision medicine. In recent years, indocyanine green (ICG), an ideal and efficient imaging medium, has been widely used in cardiovascular angiography, liver function evaluation, and other fields by its fluorescence characteristics. Fang et al. used the independently developed 3D laparoscopic AR-based surgical NS to project the preoperative 3D model to the surgical field of vision and performed 3D laparoscopic hepatectomy combined with ICG fluorescence fusion image navigation. The amount of blood loss and the blood transfusion rate in the navigation group were lower than those in the nonnavigation group. The application of AR navigation technology combined with ICG molecular fluorescence imaging in 3D laparoscopic hepatectomy has shown a good therapeutic effect. Some studies have applied ICG to evaluate the blood perfusion of the residual pancreas and reconstructed anastomosis during pancreaticoduodenectomy. If ICG is used together with 3D reconstruction in pancreatic surgery, the safe operation can be performed.
Currently, 3D reconstruction has not achieved real-time 3D reconstruction in surgery. The da Vinci surgical system can present 3D images of the field to surgeons, but its disadvantage is that it cannot see the shape of important structures. The accuracy and safety of surgery can be further improved if the 3D images of preoperative pancreatic tumor location and surrounding important blood vessels can be perfectly fused into the 3D field of vision of the da Vinci surgical system so that the important structures in the pancreas can be seen and the operation can be conducted in real-time.
Three-dimensional reconstruction, which conforms to the habit of human eyes and has the advantages of 3D visualization, can be used in pancreatic surgery to achieve preoperative accuracy, individualized surgical plan formulation, and more accurate intraoperative positioning, as well as postoperative complication prediction and surgical teaching. Three-dimensional reconstruction of the pancreas has its shortcomings, such as incomplete automated reconstruction, the inability to be used as a physical object in clinical practice, errors in the real surgical field, and the inability to achieve real-time intraoperative 3D reconstruction. However, with the progress of AI, the development of imaging media, the innovation of materials science, and the updating of surgical robots, we believe that a more stable and accurate 3D reconstruction system of the pancreas can be constructed to promote the development of pancreatic surgery.
Conceptualization: YY; data collection and analysis: YZ, GH; manuscript drafting: YZ, YY; writing review and editing: CS, JJ; and supervision: HH. All authors approved the final version of the manuscript.
This work was supported by grants from the National Natural Science Foundation of China (No. 82001895 and 82072074) and the Joint Funds of Scientific and Technological Innovation Program of Fujian Province (grant number 2020Y9064). None of the funding bodies play any role in the study other than to provide funding.
Conflicts of interest
The authors declare no conflicts of interest.
. Chessa M, Van De Bruaene A, Farooqi K, et al. Three-dimensional printing, holograms, computational modeling, and artificial intelligence for adult congenital heart disease care: an exciting future. Eur Heart J 2022;43:2672–2684.
. Saito Y, Sugimoto M, Imura S, et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg 2020;271:e4–e7.
. Shi B, Huang H. Computational technology for nasal cartilage-related clinical research and application. Int J Oral Sci 2020;12:21.
. Xiao AY, Tan ML, Wu LM, et al. Global incidence and mortality of pancreatic diseases: a systematic review, meta-analysis, and meta-regression of population-based cohort studies. Lancet Gastroenterol Hepatol 2016;1:45–55.
. Karunakaran M, Barreto SG. Surgery for pancreatic cancer: current controversies and challenges. Future Oncol 2021;17:5135–5162.
. Marconi S, Pugliese L, Del Chiaro M, et al. An innovative strategy for the identification and 3D reconstruction of pancreatic cancer from CT images. Updates Surg 2016;68:273–278.
. Won CT, Wook CJ, Hyo-Cheol K et al. Anatomic variations of the hepatic artery in 5625 patients. Radiol Cardiothorac Imaging. 2021;3:e210007.
. Sitarz R, Berbecka M, Mielko J, et al. Awareness of hepatic arterial variants is required in surgical oncology decision making strategy: case report and review of the literature. Oncol Lett 2018;15:6251–6256.
. Tomimaru Y, Tanaka K, Noguchi K, et al. The branching pattern of the inferior pancreaticoduodenal artery in patients with a replaced right hepatic artery. Am Surg 2018;84:796–800.
. Miyamoto R, Oshiro Y, Sano N, et al. Three-dimensional surgical simulation of the bile duct and vascular arrangement in pancreatoduodenectomy: a retrospective cohort study. Ann Med Surg (Lond) 2018;36:17–22.
. Miyamoto R, Oshiro Y, Nakayama K, et al. Three-dimensional simulation of pancreatic surgery showing the size and location of the main pancreatic duct. Surg Today 2017;47:357–364.
. Sakaguchi T, Suzuki S, Morita Y, et al. Analysis of anatomic variants of mesenteric veins by 3-dimensional portography using multidetector-row computed tomography. Am J Surg 2010;200:15–22.
. Ishikawa Y, Ban D, Matsumura S, et al. Surgical pitfalls of jejunal vein anatomy in pancreaticoduodenectomy. J Hepatobiliary Pancreat Sci 2017;24:394–400.
. Wang L, Wu D, Cheng YG, et al. Warshaw technique in laparoscopic spleen-preserving distal pancreatectomy: surgical strategy and late outcomes of splenic preservation. Biomed Res Int 2019;2019:4074369.
. Moekotte AL, Lof S, White SA, et al. Splenic preservation versus splenectomy in laparoscopic distal pancreatectomy: a propensity score-matched study. Surg Endosc 2020;34:1301–1309.
. Zhou HT, Peng CB, Han Y, et al. Quantitative analysis of three-dimensional reconstruction data to guide the selection of methods for laparoscopic distal pancreatectomy. J Hepatobiliary Pancreat Sci 2021;28:659–670.
. Mizrahi JD, Surana R, Valle JW, et al. Pancreatic cancer. Lancet 2020;395:2008–2020.
. Fang CH, Kong D, Wang X, et al. Three-dimensional reconstruction of the peripancreatic vascular system based on computed tomographic angiography images and its clinical application in the surgical management of pancreatic tumors. Pancreas 2014;43:389–395.
. Fida B, Cutolo F, di Franco G, et al. Augmented reality in open surgery. Updates Surg 2018;70:389–400.
. Onda S, Okamoto T, Kanehira M, et al. Identification of inferior pancreaticoduodenal artery during pancreaticoduodenectomy using augmented reality-based navigation system. J Hepatobiliary Pancreat Sci 2014;21:281–287.
. Yasuda J, Okamoto T, Onda S, et al. Novel navigation system by augmented reality technology using a tablet PC for hepatobiliary and pancreatic surgery. Int J Med Robot 2018;14:e1921.
. Okamoto T, Onda S, Yanaga K, et al. Clinical application of navigation surgery using augmented reality in the abdominal field. Surg Today 2015;45:397–406.
. Eid M, De Cecco CN, Nance JW Jr, et al. Cinematic rendering in CT: a novel, lifelike 3D visualization technique. AJR Am J Roentgenol 2017;209:370–379.
. Hafezi-Nejad N, Fishman EK, Zaheer A. Imaging of post-operative pancreas and complications after pancreatic adenocarcinoma resection. Abdom Radiol (NY) 2018;43:476–488.
. Rowe SP, Chu LC, Fishman EK. Initial experience with 3D CT cinematic rendering of acute pancreatitis and associated complications. Abdom Radiol (NY) 2020;45:1290–1298.
. Hong SM, Jung D, Kiemen A, et al. Three-dimensional visualization of cleared human pancreas cancer reveals that sustained epithelial-to-mesenchymal transition is not required for venous invasion. Mod Pathol 2020;33:639–647.
. Bailer R, Martin RCG II. The effectiveness of using 3D reconstruction software for surgery to augment surgical education. Am J Surg 2019;218:1016–1021.
. Lin C, Gao J, Zheng H, et al. Three-dimensional visualization technology used in pancreatic surgery: a valuable tool for surgical trainees. J Gastrointest Surg 2020;24:866–873.
. Han XF, Laga H, Bennamoun M. Image-based 3D object reconstruction: state-of-the-art and trends in the deep learning era. IEEE Trans Pattern Anal Mach Intell 2021;43:1578–1604.
. Sharma S, Kumar V. 3D Face Reconstruction in Deep Learning Era: A Survey. Arch Comput Methods Eng 2022;29:3475–3507.
. Huber T, Huettl F, Tripke V, et al. Experiences with three-dimensional printing in complex liver surgery. Ann Surg 2021;273:e26–e27.
. Bartel T, Rivard A, Jimenez A, et al. Medical three-dimensional printing opens up new opportunities in cardiology and cardiac surgery. Eur Heart J 2018;39:1246–1254.
. Yang YY, Huang HG. Development status of three-dimensional printing technology in pancreatic surgery. Zhonghua Wai Ke Za Zhi 2017;55:795–797.
. Yang YY, Zhao CQ, Wang LS, et al. A novel biopolymer device fabricated by 3D printing for simplifying procedures of pancreaticojejunostomy. Mater Sci Eng C Mater Biol Appl 2019;103:109786.
. Zhang P, Luo H, Zhu W, et al. Real-time navigation for laparoscopic hepatectomy using image fusion of preoperative 3D surgical plan and intraoperative indocyanine green fluorescence imaging. Surg Endosc 2020;34:3449–3459.
. Rho SY, Kim SH, Kang CM, et al. Is ICG-enhanced image able to help predict pancreatic fistula in laparoscopic pancreaticoduodenectomy? Minim Invasive Ther Allied Technol 2019;28:29–32.
. Zwart MJW, Jones LR, Balduzzi A, et al. Added value of 3D-vision during robotic pancreatoduodenectomy anastomoses in biotissue (LAEBOT 3D2D): a randomized controlled cross-over trial. Surg Endosc 2021;35:2928–2935.