Role of artificial intelligence in hepatobiliary and pancreatic surgery

Hassaan Bari,Bobby V M Dasari,Department of HPB and Liver Transplantation Surgery,Queen Elizabeth Hospital,Birmingham B15 2TH,United Kingdom

Sharan Wadhwani,Department of Radiology,Queen Elizabeth Hospital,Birmingham B15 2TH,United Kingdom

Abstract Over the past decade,enhanced preoperative imaging and visualization,improved delineation of the complex anatomical structures of the liver and pancreas,and intra-operative technological advances have helped deliver the liver and pancreatic surgery with increased safety and better postoperative outcomes.Artificial intelligence (AI) has a major role to play in 3D visualization,virtual simulation,augmented reality that helps in the training of surgeons and the future delivery of conventional,laparoscopic,and robotic hepatobiliary and pancreatic (HPB) surgery;artificial neural networks and machine learning has the potential to revolutionize individualized patient care during the preoperative imaging,and postoperative surveillance.In this paper,we reviewed the existing evidence and outlined the potential for applying AI in the perioperative care of patients undergoing HPB surgery.

Key Words:Artificial intelligence;Liver surgery;Pancreatic surgery;Augmented reality;Virtual reality;Intra-operative

INTRODUCTION

Outcomes of hepatobiliary and pancreatic (HPB) surgery have improved tremendously over the past decade with reduced postoperative mortality from 20% to less than 3% and 5%-6% for major liver and pancreatic surgery,respectively[1,2].Such an improvement has been attributed to more sophisticated preoperative imaging and improved perioperative care,progressive surgical techniques with a better anatomical understanding of anatomy,technological advancement of intra-operative instrumentation,early identification,and management of complications[3,4].However,procedures remain technically complex,requiring careful preoperative planning,intraoperative execution from experienced surgeons,anesthetists,nursing staff,and longer operative hours.The postoperative morbidity remains high at 20%-30%,and mortality rates for some of the more complex resections is reported to be as high as 10%[5-8].There is a need to continue to explore and integrate the novel and innovative technological tools into the clinical practice to improve these outcomes.

Artificial Intelligence (AI) has been investigated for its role in predictive population risk stratification,clinical decision support systems,promoting it into the new era of digital medicine,and precise surgery[9-12].Most of the uses of AI are based on machine learning,which is a technique that can automatically learn,recognize specific patterns,and make useful decisions based on the available data[13].Deep learning is part of the same technique,which replicates the neural network of the human brain for data analysis.The most representative characteristic of deep learning is that it is based on actual data,and the decision process is accomplished with minimal human interventions[14,15].Integration of such processes into the various aspects of delivery of HPB surgery will allow improving oncological and post-operative outcomes.

In this article,we review the existing and the future role of AI in HPB surgery by focusing on (1) preoperative planning [three-dimensional (3D) visualization and printing];(2) intra-operative care;integrated use of augmented reality in open and minimally invasive (laparoscopic and robotic) surgery;and (3) finally its application in postoperative care and radiomics (Table1).

AI-BASED PREOPERATIVE IMAGING

3D visualisation and virtual simulation

Currently,most HPB surgeons use two-dimensional (2D) images from computed tomography (CT),magnetic resonance imaging (MRI) scans to evaluate the position of a lesion,and its relationship to the surrounding structures in the preoperative planning.However,the 3D cognitive interpretation of spatial structure of the tumor and its relation with surrounding structures can be misjudged at times.Professor Marescaux[16]was the first to use 3D visualization to delineate the complex liver anatomy in 1998.3D reconstruction of 2D images from CT scan and MRI helps the surgeon to visualize the spatial relationship of the tumor and surrounding intrahepatic structures,identify the normal vascular and biliary anatomy and its variations,and ultimately improves preoperative planning and at times to be able to consider surgical resection based on 3D imaging in patients considered unresectable on 2D images or vice versa[17-19].Fanget al[18,20]demonstrated that patients who underwent surgery based on a 3D operation plan had lesser operation time (P= 0.028),lower hepatic inflow occlusion (P= 0.029),and reduced high grade (Clavien-Dindo grade III-V)postoperative complications (P= 0.048) as compared to patients who underwent surgery without any 3D planning[18,20].

Hilar cholangiocarcinoma usually presents in an advanced stage when the caudate lobe and hilar structures are already invaded[21].These patients require radical resection with hepatectomy,associated with significantly increased mortality of 10%-15% and a significantly higher postoperative morbidity of up to 40%[22,23].Zhanget al[24]in a study of 23 patients with hilar cholangiocarcinoma of Bismuth grades III and IV,showed that preoperative 3D reconstruction could accurately determine the presence of tumor invasion into hilar vessels,variant hilar anatomy,and future liver volumesthat helped to develop an individualized operative plan for a patient[24].Similar benefits were reported by Okudaet al[25]along with higher negative resection margins for biliary malignancies for patients who underwent preoperative surgical planning with 3D reconstruction[25].Another benefit of 3D reconstruction is the accurate stereoscopic assessment of portal vein anatomy and determining the line of parenchymal transection and planning portal vein reconstruction[26].Other studies also reported similar benefits including reduced amount of intraoperative bleeding with the use of preoperative 3D reconstruction[27-29].

Table1 Summary of the studies included in the review evaluating the role of artificial intelligence in hepatobiliary and pancreatic surgery

3D visualization techniques are also used to perform virtual liver resection before actual surgery to assess the resectability of the lesion and calculate future liver remnant.In patients with hepatocellular carcinoma (HCC),virtual hepatectomy allowed more aggressive surgery based on portal territory-oriented resection with a higher disease-free 5-year survival.Similarly,patients with colorectal liver metastasis(CRLM) who underwent virtual hepatectomy had equivalent long term outcomes to patients who did not have a virtual resection,despite the larger tumor load in the virtual hepatectomy group[30].Virtual hepatectomy is also of great use in living donor liver transplantation procedures.The donor selection can be further optimized based on the information available from 3D imaging,including the need for venous reconstruction based on the donor vascular anatomy resulting in improved safety of operation[30].

3D visualization of peri-pancreatic vessels before surgery is reported to have reduced the operative time,blood loss,and hospital stay significantly compared to patients who underwent surgery based on 2D image planning prior to pancreaticoduodenectomy (P= 0.024) and distal pancreatectomy (P= 0.026)[31,32].Fanget al[33]reported sensitivity and sensitivity of 100% for resectability assessment of pancreatic cancer by 3D reconstruction[33].It also helps determine the size and location of the main pancreatic duct before surgery,which may help select optimal anastomotic technique[31].Assessment of resectability of the borderline resectable tumors (with involvement of surrounding vessels) is vital as 25% of the patients explored surgically are considered unresectable at laparotomy.It is likely that the benefits of 3D reconstruction be translated to improve the resection rates in this group.

3D printed models

Although 3D images and reconstruction can significantly benefit the understanding the surgical anatomy,the display is still on 2D screen,limiting its use.This limitation can be overcome by converting 3D reconstructed images into real physical models with 3D printing technology.The first use of 3D printing was reported in 2013 by Zeinet al[34].Since then,3D printing has been reported to be useful in treating liver tumors and also in liver transplants[35-39].One substantial benefit of the 3D printed model is that they can be brought into the operating room and compared with the real liver during surgery and adjusted in the optimal anatomical position to identify intrahepatic structures.This advantage of navigating on a real physical liver model is that it can locate small,disappearing CRLM and perform precise segmentectomies.3D-printed models allow visualization and planning of the exact line of transection[36,39].Xianget al[26]reported the ability to precisely identify and manage the replaced hepatic arteries,segment IV portal vein branch coming from the right anterior portal vein to plan a right hemihepatectomy with extreme precision and prevent segment IV ischemia in a patient with HCC and avoided post hepatectomy decompensation of the liver function[26].Burdallet al[38]reported using a hybrid 3D model containing hepatic,pancreatic,and choledochal components and used it to simulate laparoscopic choledochal surgery[38].In living donor liver transplantation for small infants and neonates,3D printed liver models have shown promising results in assessing the size discrepancies between recipient and graft[39].Such information is useful for an adequate plan to reduce the graft volume and complex vascular and biliary structures of the liver than traditional CT imaging and help avoid unexpected surgical complications while preserving the vital vascular structures[39].However,such a comparison of the real-time and printed models will need 1:1 size matching increasing the production cost of each model.

Low-cost 3D liver printed models were used for trainees' medical education and to help them practice hepatectomy operations with a positive impact on the understanding of liver anatomy,better visualization,and higher learning efficacy[40,41].The recent 3D Bio-printed models consist of new-generation bio-inks and hepatic cells,which are biocompatible,scalable,convenient,and low-cost[42].These models have a vital role in developing liver tissue engineering and even artificial liver,which may expand their therapeutic role in managing patients with liver failure[42].

AI can reduce the complexity of a traditionally manual process of 3D printing.The main application of AI in 3D printing comes from automation of workflow.This comprises various steps,from the creation of the model as a Computer Aided Design file,to its preparation for printing in a slicing software,to its final printing.AI can also help improve the 3D printing process by assessing the printability before starting any process.The quality of the final product can also be predicted and the process be controlled to avoid printing errors,effectively saving time.

Intraoperative use of AI in HPB surgery

The major drawback of 3D reconstruction and printing techniques lies in associating the 3D reconstructed images or physical models to the actual surgery due to inadequate synchronization between the two modalities[43].These limitations could be overcome using computer guiding software,which combines preoperative 3D reconstructive images with intraoperative information in real-time.It can be done by Augmented Virtuality (AV) that displays a virtual environment that is controlled by real information or by Augmented Reality (AR) that displays virtual information based on real images of the patient.It is a relatively new and unused tool to improve oncological safety in the field of HPB surgery.It can confirm the ideal dissection plane and anatomical landmarks in real-time and help achieve safe margins with maximum functional preservation[44].AV,AR,and mixed reality (MR) offer a safe and reliable surgical navigation method,which reduces the chances of misinterpretation between the 3D reconstruction model and actual operating space[30,44].

For locally advanced pancreatic cancers,intraoperative computer-based navigation can be used to assess the spatial relationship of the tumor with the involved vessels and the possibility of venous reconstruction or arterial involvement depending on tumor invasion,increasing the safety,and effectiveness of the procedure[45].Okamotoet al[46]reported 19 patients who underwent AR-based navigation surgery for pancreatectomy[46].In this study,reconstructed preoperative images were superimposed on the real organs on the monitor display during surgery,and it corresponded to that of the actual organs[46].Such information is most useful in patients with small pancreatic neuroendocrine tumours,liver lesions deep in the parenchyma that are challenging to navigate at minimally invasive surgery,and planning of microwave coagulation therapy[47-49].Modern chemotherapy for CRLM can cause shrinkage of tumors to the extent that they may disappear on postchemotherapy scans.In a pilot study,Ntourakiset al[49]reported that AR helped in detecting those missing lesions and achieving a negative margin with no local recurrence at a median follow up of 22 mo[49].

AI in minimally invasive HPB surgery

Laparoscopic cholecystectomy is one of the most common general and speci alist surgical procedures performed worldwide.Injury to common bile duct is considered one of the avoidable complications that is otherwise associated with the need for further interventions and adds a considerable burden of medico-legal litigation.The risk of bile duct injury can be reduced by correctly identifying the standard anatomical landmarks at surgery (the common bile duct,cystic duct,lower edge of the left medial liver segment,Rouviere's sulcus).Tokuyasuet al[50]developed an AI-based learning model to detect these four anatomical landmarks using real-time object detection algorithms on using a training set of over 2000 endoscopic camera images[50].These landmarks were successfully identified with adequate precision intra-operatively by the validation cohort,and such novel systems can reduce bile duct injury during laparoscopic cholecystectomy and other iatrogenic injuries[50].

Minimally invasive surgery for major laparoscopic hepatectomies is being practiced routinely in specialist units with reported benefits of reduced scar size,postoperative morbidity,and shorter hospital stay[51].Donor hepatectomy,pancreaticoduodenectomy are also increasingly performed by experienced specialist surgeons by minimally invasive means.However,in addition to the long learning curve for the surgeons to be able to perform these procedures,one of the main issues is the loss of the tactile sensation,which is replaced by force feedback through a laparoscopic instrument to differentiate between the tissues of varying consistency,making it challenging to appreciate tumor margins and be able to identify the smaller but vital vascular structures.

For minimally invasive surgeries,it seems appropriate to use AR techniques to superimpose in the endoscopic view structures which are not visible by direct camera view but are visible in the preoperative images[52,53].AR technology uses CT and MRI data to reconstruct a 3D image of the liver and detailed intrahepatic vasculature,and the virtual image is superimposed on the liver surface in a 1:1 ratio,to assist the operating surgeon during surgery[53,54].Phutaneet al[55]showed that AR-based hepatectomy for HCC could help detect intrahepatic tumors,the transection plane,and locating the hepatic veins before parenchymal transection,which can reduce bleeding and duration of surgery[55].Similar findings were reported by Halletet al[56]who used tans-thoracic minimally invasive liver resection guided by AR[56].

Loss of depth perception with monocular cameras is also another disadvantage contributing to the longer operative times[57,58].These limitations could potentially be overcome with the help of AR and MR technology,which will not only aid in finding the position the intra-parenchymal lesions but also provides a better field of vision.This facilitates the oncological resection and limits the risk of operative bleeding[30,59].It also provides a solution to reduce the gap between the three-dimensional reconstruction model and the actual operating space,which helps overcome uncoordi-nated hand and eye maneuvering during surgery[43,54].

AR also has a vital role in laparoscopic surgical education of trainees.For surgeons in the earlier part of the career,this could reduce the duration of learning curves.A recent randomized controlled trial showed that AR-based training could improve the necessary skills of laparoscopic cholecystectomy in surgical residents and overcome the learning curve[60].It has distinct advantages and broad prospects in many aspects,such as preoperative planning,intraoperative navigation,surgical education,and doctor-patient communication[54].Although 3D visualization and AR allow further navigation,there is still a need to develop more on the haptics to replace the palpation,and tactile sensation in minimally invasive HPB surgery,may it be to determine the boundaries between normal and cancer tissue.

The limitations include that the patient-specific 3D reconstructed images need to be prepared using complex algorithms requiring a lot of time and effort.This problem can be overcome by real-time acquisition of high-resolution preoperative scans and 3D reconstructions.Secondly,preparing the whole system to achieve the desired navigation during surgery and required registrations;the process itself can significantly increase the overall duration of surgery and anesthesia time for patients.It varies on the type of procedure and the complexity of the AR system as well.The solution to this problem is to develop fully automated systems,which would reduce the total time required for completion.However,to date,most of the registration is performed manually.

Robotic HPB surgery

The use of AI techniques in HPB robotic surgery helps achieve exceptional performance with increased ability to perform the fine skills in delineating complexin vivohilar and pancreatic anatomy,helping the operating surgeon make accurate decisions and perform the desired task with increased meticulousness and efficiency[12].3D imaging,multi-fold magnification,and significantly improved dexterity are the most noticeable features of currently available robotic systems that allow precise tumor localization,dissection,reduce blood loss,and potentially higher success rates for certain types of hepatobiliary procedures like spleen-preserving distal pancreatectomies compared to a traditional open approach[61].10-fold magnified 3D intra-operative views of robotic surgery overcome the limitation of depth perception associated with the laparoscopic technique.It helps in the dissection of delicate tissue like liver parenchyma,and the increased dexterity,even in narrow spaces,allows for intra-corporeal suturing at the same time[62].This was highlighted in a matched comparative study in which higher rates of successful,purely minimally invasive approaches were reported with the robotic technique compared with conventional laparoscopy for major hepatectomies[63].As with laparoscopy,the surgeon operating with the help of a robot cannot use the "sense of touch" to identify blood vessels or differentiate between healthy and scar tissue by manual palpation.

AR has been used to overcome this limitation[58].Pessauxet al[64]reported the use of the see-through visualization feature of AR for port placement,one of the most crucial steps in robotic surgery.Each robotic port was placed according to the patient's anatomy,variations,and target lesions.AR allowed for the accurate and safe identification of intrahepatic vascular structures throughout the surgery.Hepatic pedicle clamping was not used in any of the cases,and none of the patients required perioperative transfusion[64].In another study,Buchset al[65]reported the benefits of AR-based robotic resection in patients who had resection of HCC.The augmented endoscopic view delineated an accurate resection margin around the tumor.The overlay of reconstructed 3D models also helped during parenchymal transection to identify vascular and biliary structures,and safe tumor margin widths of 0.5 cm and 1 cm were achieved,with no complications[65].Constant research and development have also enabled robots to automatically perform somein vitrosimple surgical errands,such as suturing and knot tying[66].However,the current equipment and technology are still far from attaining complete autonomy to robots in surgery,and human control would continue for safety and complex decision-making.

Regarding intraoperative limitations,inattentional blindness,an event when an unexpected object suddenly appears in the surgeons' field of view,is one of the biggest concerns that need to be addressed while using 3D overlays[67].AR provides a vast amount of data and information to surgeons,which may be distracting[68].Therefore,it is of utmost importance to project only relevant data or develop a method to display different sets of information depending on surgeons' needs.The latency of the whole system of AR is also a concern.Currently,the latency for laparoscopic procedures is reported to be 144 ± 19 milliseconds[69],and any prolonged latency in robotic surgery is best avoided to maintain the accuracy and the surgeon's comfort.

PREDICTIVE MODEL GUIDED POSTOPERATIVE MANAGEMENT

Predictive models for postoperative morbidity

HPB surgery is associated with high postoperative morbidity and mortality rates.The reported morbidity rate for major hepatectomy is 25%,and for pancreaticoduodenectomy,it is nearly 40%[5-8,70].Early prediction of morbidity with detailed attention and thorough postoperative management of complications can positively impact the overall outcomes of complex HPB procedures.The perioperative imaging information can be combined with clinical data to establish corresponding diagnostic and predictive models.For instance,a machine learning technique was applied to develop an algorithm after extracting data of 15657 patients was used to predict the postoperative morbidity after HPB and colorectal surgery[71].This algorithm had a better predictive (C-statistic of 0.74) ability than other established methods like the American College of Surgery-risk calculator (C-statistic 0.71) and American Society of Anaesthesiologists levels (C-statistic 0.58).The algorithm had excellent performance in predicting postoperative stroke,wound dehiscence (superficial SSI,organ space SSI),cardiac arrest,and progressive renal failure (C-statistic 0.96-0.98).The algorithm also showed a good predictive ability for sepsis and perioperative hemorrhage[71].Similarly,algorithms are being used to predict the risk of liver failure after hepatectomy[72,73].Patients at risk of liver failure may avoid major hepatectomy and undergo adjuvant or an alternative treatment.

Predictive models for a liver transplant

In liver transplant,data that predicts the expected survival with a specific graft is crucial in selecting the best recipient for each graft.Wingfieldet al[74]in a systematic review suggested that AI based predictive model can be used to determine graft outcomes after deceased donor liver transplant.These models were based on multiple factors from donor,recipient,and graft.The AI-based models proved to be superior in predicting graft survival than the traditional log regression model and other classic scores (MELD,SOFT)[74].Similar findings were reported by Liuet al[75]when they developed a machine learning-based algorithm to predict short term survival after liver transplant[75].These models can be used to assign the graft to appropriate recipients and improve survival and the outcomes after liver transplant.

Radiomics in HPB surgery and models for cancer surveillance

Radiomics presents a new horizon to generate new understanding and concepts within specific areas of pathology.It is based on the techniques which mines quantitative data from patients imaging (Ultrasound,CT,MRI,and PET/CT) and analyses it to retrieve clinically relevant information that can be used for diagnosis and prognostication[76].For example,Parket al[77]recently developed a radiomics fibrosis index to assess liver cirrhosis[77].The model was based on data extracted from MRI images of the liver.This radiomics index demonstrated to be considerably better than routine normalized liver enhancement and serum fibrosis indices[77].A similar CT based radiomic model was used for the diagnosis and severity of portal hypertension.This model was significantly better in diagnosing portal hypertension than clinical indices and other methods like liver stiffness[78].

The role of radiomics is getting significantly crucial in the field of hepatobiliary oncology.The role is postulated on the theory that a radiologic phenotype can imitate genetic variations of carcinogenesis and help determine the expected tumour behavior.These radiomics based models are being used to substantiate clinical decision making and practice precision medicine.Hepatocellular carcinoma is associated with a high recurrence rate.Early recurrence after resection is itself a poor prognostic factor,reducing 5-year survival rates from 70% to 30%[79].Almost 50% of patients develop recurrence within five years of surgery.If patients at increased risk of recurrence are identified using accurate algorithms,the physician can arrange more close surveillance.Zhouet al[80]developed a model based on 21 radiomic features from 300 patients[80].This model proved that combining conventional clinical factors and radiomics feature can perform better in accurately predicting early recurrence than with Barcelona Clinic Liver Cancer (BCLC) staging,Childs classification,and other clinical features alone[80].Microvascular invasion is one of the indicators of early recurrence in patients with HCC[81].This information is often available only on postresection histological analysis.Banerjeeet al[82]developed a model based on a cluster of preoperative radiomic features for predicting the presence of microvascular invasion[82].Its accuracy can reach up to 94%,which is better than the results based on imaging only[82].Zhenget al[83]developed a CT-based radiomic normogram to predict recurrence-free survival rates for HCC after resection,ablation,and transplant.These normograms predicted the prognosis and recurrence much more effectively than the traditional staging[83].

Elarreet al[84]evaluated the 2-year relapse risk for pancreatic cancer patients based on a machine-learning algorithm[84].The main goal was to provide prognostic information to patients who underwent pancreatic resection.This model showed an accuracy of more than 60% for disease recurrence within two years of surgery[84].It proved to be a valuable tool,especially for high-risk patients.Intensive surveillance and extended use of adjuvant treatment for such patients can be considered based on this model[84].

CONCLUSION

The ultimate goal of AI is to achieve a better and individualized healthcare plan for each patient.Integrating the genomic and molecular targeting information and clinicpathological features of the individual liver and pancreatic cancer patients will enable surgeons to provide precise and personalized surgery with the aid of surgical technology enhanced with 3D imaging,AR,VR and MR modalities.