Although gastrointestinal endoscopy can remove most foreign bodies in the gastrointestinal tract, in approximately 1% of cases the foreign body cannot be found and removed by endoscopy because it is in a difficult location or has been in place a long time, and surgical treatment is still required [1]. Even if the pancreas is located deep in the abdomen, ingested foreign bodies may penetrate the digestive tract and enter the pancreas or even be completely enclosed in the parenchyma [2,3,4]. They may also cause some complications, such as pancreatitis or pancreatic abscess and bleeding, making surgical removal more challenging.

If the foreign bodies completely penetrate the pancreas, one of the difficulties of surgical resection is determining the precise position of the foreign body during surgery. With two-dimensional images such as preoperative CT or ultrasound, it is difficult for surgeons to stereoscopically determine the location of foreign bodies. Therefore, surgeons may need to perform a large-area dissection of the pancreas and surrounding tissues to find the target. This process may cause secondary injuries such as bleeding and pancreatic leakage, especially in patients with long-term foreign body residues leading to vascular cavernous transformation and local venous hypertension [5, 6].

Emerging technologies such as three-dimensional (3D) visualization technology and augmented reality navigation (ARN) have been widely used in hepatobiliary surgery [7,8,9]. Individualized surgical approaches can be repeatedly practiced on the 3D models to select the most suitable surgical approach. By combining preoperative 3D models and intraoperative laparoscopic imaging, ARN can enable the visualization of non-apparent structures [10, 11]. In addition, dynamic fusion can provide continuous real-time guidance in ARN and help surgeons accurately identify important blood vessels and other anatomical structures during surgery to facilitate the safe removal of foreign bodies [12].

To our knowledge, this is the first report that ARN has been innovatively applied in laparoscopic removal of a foreign body in the pancreas and achieved good clinical results.

Patient and methods

This study was approved by the medical ethical committee of Zhujiang Hospital and was conducted according to the relevant guidelines regulations. The patient was informed and agreed to the contents of this study.

A 37-year-old female had epigastric pain for 3 months and had been treated as stomach illness. However, her symptoms did not improve, so she came to our center for treatment. The abdominal CT showed a linear high-density shadow in her pancreas along with chronic pancreatitis. The confluence of the superior mesenteric vein and splenic vein was occluded, causing cavernous transformation of the portal vein (Fig. 1A). Considering that she has a history of eating fish, we believed that this was a chronic disease caused by fish bone penetrating the posterior wall of the antrum and entering the pancreas. Because it had been in place a long time and the fish bone were was not in the gastrointestinal tract, endoscopic removal was excluded. A secure laparoscopic surgery was needed. However, the dilemma of surgery was how to accurately locate the fish bone in the pancreas instead of blindly probing to avoid unnecessary destruction of the pancreatic parenchyma and varicose vascular injury caused by large-scale dissection.

Fig. 1
figure 1

A CT showed a linear high-density shadow in the pancreas and chronic pancreatitis. B The 3D model showed the positional relationship between the foreign body and the surrounding organs and blood vessels. C The 3D model showed portal vein cavernous transformation and vascular masses of splenogastric varicosities. D Application of intraoperative ultrasound and augmented reality navigation system during the operation

Full size image

3D models of the liver, pancreas, stomach, blood vessels, and fish bone were created using the Medical Image 3D Visualization System (MI-3DVS, software copyright No. 2008SR18798) based on preoperative CT images. The 3D models were amplified, rotated, and hyalinized to identify the location of fish bone and the anatomic character of the tissue structure [13]. As shown in the 3D model, the foreign body was enclosed completely in the neck of pancreas above the gastric antrum (Fig. 1B, C). Gastroptosis was found, so the surgical approach was adapted to open the hepatogastric ligament to reach the pancreas instead of cutting the gastrocolonic ligament.

The patient was positioned in the supine position with legs separated. The operating surgeon stood on the right side of the patient, while the assistant stood on the left side and another assistant holding the laparoscope stood at the foot (Fig. 1D). After the creation of a pneumoperitoneum, routine abdominal cavity exploration was performed. Then, the hepatogastric ligament was severed but the foreign body was still invisible. Because the location of the foreign body was obscure, it was challenging to completely locate its stereoscopic structure. We choose to use ARN assistance instead of only intraoperative ultrasound (IOUS) to find the position of the fishbone as quickly as possible.

After 2–3 s of video images were captured by 3D laparoscopy, an intraoperative image was imported into the ARN system and a 3D dense stereo-reconstruction method was used to obtain the surface model of pancreas, stomach, and blood vessels. In order to register the pancreas, stomach and blood vessels and the preoperative 3D models, the Globally Optimal Iterative Closest Point (Go-ICP) method was used to obtain a spatial transformation matrix between the preoperative CT image space and the intraoperative laparoscopic space (Fig. 2A) [9, 14, 15]. A manual registration method is available if the automatic registration result is not satisfactory. The subsequent fusion dynamically adjusted the preoperative and intraoperative spatial transformation matrix according to the initial registration results to track the laparoscopic posture modules. The ORB-SLAM2 method was used to estimate the position of the laparoscopic camera as well [16]. Under ARN guidance, the position and location of the foreign body were displayed on the surface of the pancreas. Then IOUS was used for further verification and to quickly and easily confirm the surgical entrance (Fig. 2B). After minimal dissection and removal of the pancreatic parenchyma, the superficial end of the foreign body was exposed and clamped. Finally, the foreign body was removed completely and the vascular injury and pancreatic leakage caused by large-scale dissociation were avoided (Fig. 2C).

Fig. 2
figure 2

A Application of ARN: The organs, blood vessels, and foreign body location were visualized. B IOUS located the superficial end of foreign body. C The foreign body was removed from the pancreas laparoscopically. D The foreign body was identified as a 3-cm-long fishbone

Full size image

Results

The operation time was 60 min, the estimated blood loss was 10 ml, and there was no need for blood transfusion. The foreign body was identified as a 3-cm-long fishbone (Fig. 2D). On the second day after the operation, liver function and serum amylase level were normal. The patient recovered without complications and was discharged on the third postoperative day.

Discussion

Fish bones are one of the most common foreign bodies that enter the digestive tract and cause perforation or penetration [17,18,19]. Foreign bodies may pass through the stomach or duodenum into the pancreas and cause complications [2, 3, 20]. Gastrointestinal endoscopy can diagnose and treat foreign bodies in the digestive tract [1]. But if the foreign body has penetrated the stomach wall and completely entered the pancreatic parenchyma, it is impossible to diagnose and remove it using endoscopy. Therefore, surgical removal is necessary. However, in this case, even with preoperative CT and ultrasound two-dimensional images, the position of the fish bone and the relationship with the surrounding tissues was difficult to accurately identify, even by experienced surgeons.

Through the segmentation and reconstruction of 3D reconstruction technology, two-dimensional images of CT are transferred into 3D visualization. The 3D reconstruction models, which provide more intuitive and omnidirectional information on pancreas, stomach, blood vessels, and foreign bodies, can overcome the limitations of CT and ultrasound images. In 3D models, the anatomic character of tissue structure can be clarified with omnidirectional, multiple-angle, and multilevel views [13]. In this case, 3D reconstruction was used to determine that the fish bone completely entered the parenchyma of pancreas and caused the expansion of the distal pancreatic duct. The occlusion of the confluence of the superior mesenteric vein and splenic vein resulted in obstruction of two venous returns, and varicose vessels were further confirmed. In addition, gastroptosis was confirmed in the 3D reconstruction, and the part of the pancreas where the fishbone was located was covered by the hepatogastric ligament. Therefore, the approach was to open the hepatogastric ligament rather than the gastrocolic ligament, in order to avoid damage to the fragile varices.

However, even with 3D reconstruction models, the internal status of the pancreas is still invisible during the operation. IOUS can enable 2D image navigation, but the acquisition of images is difficult and unstable. In addition, it requires the surgeon to move the visual focus between the laparoscopic surgery system, the 3D model display system, and the intraoperative ultrasound system. ARN can integrate more stable 3D images into the surgical procedure more effectively, because it allows for better coordination between images and actions and can enable the visualization of relevant structures without the need for surgeons to shift their attention constantly. Surgeons and assistants can observe overlapping images and surgical sites on the display screen of the ARN system to eliminate the hand–eye coordination problem due to the spatial and temporal separation of 3D image-guided surgery to a certain extent. In addition, this is also an effective way to facilitate team collaboration and to prevent distraction.

Another advantage of ARN is the short time for registration and visualization. After acquiring the laparoscopic video for 2–3 s, the 3D model image can be projected to obtain the anatomical features of the intraoperative structure. Compared with traditional intraoperative ultrasound navigation, ARN greatly reduces the time required to find the target and the necessary skill requirements for the operator. Moreover, subsequent fusion can be dynamically adjusted according to the camera’s position in real time. There is no need to hold the probe as long as intraoperative ultrasound requires, so surgeons hands are free.

In this study, augmented reality intuitively provides virtual transparency and allows the visualization of otherwise non-apparent structures such as the fish bone and pancreatic duct inside the pancreas, thus enabling highly accurate navigation with an error threshold below 2 mm. Dissection planes and resection margins of the pancreatic parenchyma can be delineated using ARN to reduce the risk of injury to invisible structures. Difficulties in determining accurate anatomy because of dense inflammatory adhesions can also be solved. Under the guidance of ARN, the surgeon can confirm the entry point to remove the foreign body quickly. In addition, ARN guidance can prevent unnecessary dissection of excessive adhesion tissue and separation of fragile varicose veins.

Because of the patient’s breathing movements, intraoperative organ deformation and tissue displacement are the main challenges in image-guided surgical navigation. However, the situation is different in the laparoscopic removal of foreign bodies in the pancreas. The positions of the pancreas and the foreign body are relatively fixed during surgery. Therefore, ARN can provide the stable and effective image guidance for laparoscopic removal of foreign bodies in the pancreas.

With the guidance of ARN, IOUS was used to further confirm the approach more accurately and quickly. Meanwhile, the accuracy of ARN has also been further verified in this study, and it provided the same information about the fish bone as that provided by IOUS. The combination of ARN and IOUS shortens operative time and reduces operative damage, such as intraoperative bleeding and pancreatic leakage.

Even though this technology has been currently used only for laparoscopic removal of a fish bone in the pancreas, it might be applied to removal of other foreign bodies and even tumors in the future.

Conclusion

Because it enables direct visual navigation via simple operation, ARN facilitates the laparoscopic removal of foreign bodies in the pancreas with accurate and rapid positioning and minimal damage.