Applying artificial intelligence to big data in hepatopancreatic and biliary surgery: a scoping review
Abstract
Aim: Artificial Intelligence (AI) and its applications in healthcare are rapidly developing. The healthcare industry generates ever-increasing volumes of data that should be used to improve patient care. This review aims to examine the use of AI and its applications in hepatopancreatic and biliary (HPB) surgery, highlighting studies leveraging large datasets.
Methods: A PRISMA-ScR compliant scoping review using Medline and Google Scholar databases was performed (5th August 2022). Studies focusing on the development and application of AI to HPB surgery were eligible for inclusion. We undertook a conceptual mapping exercise to identify key areas where AI is under active development for use in HPB surgery. We considered studies and concepts in the context of patient pathways - before surgery (including diagnostics), around the time of surgery (supporting interventions) and after surgery (including prognostication).
Results: 98 studies were included. Most studies were performed in China or the USA (n = 45). Liver surgery was the most common area studied (n = 51). Research into AI in HPB surgery has increased rapidly in recent years, with almost two-thirds published since 2019 (61/98). Of these studies, 11 have focused on using “big data” to develop and apply AI models. Nine of these studies came from the USA and nearly all focused on the application of Natural Language Processing. We identified several critical conceptual areas where AI is under active development, including improving preoperative optimization, image guidance and sensor fusion-assisted surgery, surgical planning and simulation, natural language processing of clinical reports for deep phenotyping and prediction, and image-based machine learning.
Conclusion: Applications of AI in HPB surgery primarily focus on image analysis and computer vision to address diagnostic and prognostic uncertainties. Virtual 3D and augmented reality models to support complex HPB interventions are also under active development and likely to be used in surgical planning and education. In addition, natural language processing may be helpful in the annotation and phenotyping of disease, leading to new scientific insights.
Keywords
INTRODUCTION
Artificial Intelligence (AI) encompasses a range of computational approaches with the central aim of developing algorithms to process and interpret information. AI methods can be applied to various input data types ranging from tabular datasets and images to multimedia and text. Although termed “intelligence”, these algorithms are in no sense conscious or able to employ “rational thought”, but in most cases, reflect model parameters derived exclusively from input data. Within AI, there are three overlapping fields that arguably have the most potential for HPB surgery: machine learning (ML), computer vision (CV) and natural language processing (NLP). ML uses algorithms to learn, adapt, and draw inferences from patterns in training data. CV allows for supervised or unsupervised image analysis, allowing for features of interest in images to be identified and characterized. For text-based sources of data written as prose or in a “human-readable” format (e.g., radiology or pathology reports), NLP allows computers to interpret human text or spoken language communication[1-5].
The specific areas and applications of AI most likely to deliver a positive impact on patient care currently need to be clarified, as are the barriers limiting the uptake of AI approaches into clinical practice. In 2021 Bari et al. described the applications of AI in hepatopancreatic and biliary (HPB) surgery, proposing the framework of preoperative, intraoperative, and postoperative AI applications. We have adopted this structure for this review[6].
With the increased availability of structured and unstructured healthcare datasets, the opportunity for AI-based approaches widens. Policymakers, healthcare providers, and industry are exploring new AI approaches, seeking to utilize data across a range of applications, including improving outcomes, optimizing the patient experience, and providing cost-effectiveness in delivering care at the health system level[7-9]. In this review, we aim to outline the fundamental AI approaches to pressing questions in HPB surgery, identifying where AI is most likely to have an impact in future patient care.
METHODS
This scoping review was performed in accordance with the PRISMA-ScR guidelines for scoping reviews[10]. The Medline database was searched systematically using the following Medical Subject Headings (MeSH) search terms to ensure the identification of appropriate articles; “Algorithms.mp. or algorithm/” AND “surgery/ or biliary tract surgery/ or liver surgery/ or pancreas surgery/”. Articles were limited to English language and those published from 2012 onwards to provide contemporary studies that were likely reflective of current approaches in AI. Further supplementary searches were performed using citation lists and the Google Scholar database. The last search was conducted on 5th August 2022.
We defined “HPB surgery”, as the surgical management of benign and malignant diseases of the liver, pancreas, gallbladder, and bile ducts. “Artificial intelligence” refers to the use of various algorithmic methods which could be applied to interpret or process information. We further assessed the identified papers for the element of AI primarily used i.e., machine/deep learning, computer vision or natural language processing[1-5].
Following the literature search, article titles and abstracts were screened by three reviewers (KMcG, SRK, JL) and those meeting the inclusion criteria underwent full-text review. Any disagreements were resolved by consensus within the group. References from included articles were searched to identify any other relevant articles. Conference abstracts were screened to assist in identifying related full-text articles before inclusion. Where more than one article was published from a single data set, the article analyzing the largest cohort of patients was included.
Data were extracted independently using a standardized pro forma. This included the aim of the study, methodology, year of publication, countries represented, the primary organ of focus, AI methods employed, and the number of patients (where applicable). Identified publications were further interrogated to find a shared focus on diagnostics, prognostics, or intervention, allowing further subdivision of the presented research. We then undertook a conceptual mapping exercise to identify areas of crucial importance.
We used a pragmatic approach to further select studies with a sample size equal to or greater than five thousand that satisfied the “velocity, volume and variety” of data points needed to be considered as “big data”. A similar approach used previously, albeit with smaller datasets, acted as a benchmark[7-9,11]. Any disagreements on the selection of these papers were resolved by group consensus.
The Covidence online toolkit was used throughout the data collection and extraction stages of this scoping review[12].
RESULTS
Scoping search results
The search identified 5,221 articles, of which 134 were fully assessed for eligibility. A further 63 articles were identified from article citation lists or by the supplementary search of the Google Scholar database [Figure 1]. Following assessment, 98 studies[13-110] were included in this review, with most studies excluded due to being in conference abstract form only (n = 84).
Figure 1. PRISMA flow chart detailing study selection and exclusion.
Characteristics of included studies
Identified studies had a wide geographical distribution, coming from a total of 24 countries, with the majority from China or the USA (45/98). No papers identified originated from the African continent [Figure 2]. Studies on the use of AI in surgical conditions of the liver predominated (n = 51). Research on pancreatic and biliary conditions (n = 23) was included at a comparable frequency to one another. We noted a rapid increase in the number of studies published over the past three years, with almost two-thirds of the identified papers (n = 61) published since 2019 [Figure 3].
Figure 2. Global distribution of studies on the use of AI in HPB Surgery.
Figure 3. Primary organ of interest over time demonstrating the increase in frequency of publication.
Studies identified were subdivided into groups focusing on diagnostics, prognostics, and interventions. We assessed 23 papers[13-35] reporting diagnostic uses of AI in HPB surgery. Of these, five focused on the gallbladder, 11 on the liver, and seven on the pancreas. Twenty-nine studies reported prognostication[36-64] using AI, of which three focused on the gallbladder, 16 on the liver, one on the liver and pancreas, and nine on the pancreas alone. Almost half of the studies identified reported on the interventional use of AI[65-110] in HPB surgery (n = 46), with 24 studies focusing on the liver, 19 on the gallbladder alone or in conjunction with another organ (n = 4), and three studies looking at the pancreas. A summary of the papers subdivided into the diagnostic, prognostic and intervention cohorts can be found in Tables 1, 2, and 3, respectively.
Summary of included studies focusing on diagnostic uses of AI in HPB surgery
| Authors | Year of publication | Location | Organ | AI method | Aim | Method | Data |
| Saftoiu et al.[13] | 2012 | Romania | P | DL/CV | Assessed accuracy of real-time EUS elastography in pancreatic lesions using artificial neural network analysis | Prospective, blinded, multicentric study | EUS images |
| Kaizhi et al.[14] | 2014 | Japan | L | DL/CV | Proposes automatic classification method based on deep learning in contrast-enhanced ultrasonography (CEUS) of focal liver lesions | Case series | CEUS images |
| Gatos et al.[15] | 2015 | Greece | L | ML/CV | Design and implementation of a computer-based image analysis system employing the support vector machine system for the classification of liver lesions | Retrospective study | MRI images |
| Roch et al[16] | 2015 | USA | P | NLP | Implement an automated Natural Language Processing based pancreatic cyst identification system | Single institution prospective pilot study | Patient records |
| Sada et al.[17] | 2016 | USA | L | NLP | Evaluated whether natural language processing document classification improves HCC identification | Retrospective study | Pathology/radiology reports |
| Kondo et al.[18] | 2017 | Japan | L | ML/CV | Proposes automatic classification method based on machine learning in CEUS of focal liver lesions | Single institution pilot study | CEUS images |
| Yang et al.[19] | 2017 | China | L | NLP | Assess gene expression in HCC using combined data from The Cancer Genome Atlas and NLP identified genes | Description of experiment | Gene library/ published literature |
| Kuwahara et al.[20] | 2019 | Japan | P | DL | Investigate whether a deep learning algorithm using EUS images of IPMN could predict the diagnosis of malignancy | Retrospective study | EUS images |
| Shen et al.[21] | 2019 | China | P | ML | Establish and validate a radiomics diagnosis model for the classification of three subtypes of pancreatic lesion | Retrospective study | CT images |
| Lei Xu et al.[22] | 2019 | China/ USA | G | ML/CV | Develop and validate a prediction model for preoperative LN status evaluation in ICC patients | Retrospective study | MRI images |
| Brown et al.[23] | 2019 | Canada | L | NLP/ML | Explore natural language processing to predict downstream radiology resource utilization in patients undergoing surveillance for HCC | Retrospective study | Radiology reports |
| Watson et al.[24] | 2020 | USA | P | DL | Use CT-guided deep learning techniques to predict malignancy of PCNs | Retrospective pilot study | CT images |
| Liu et al.[25] | 2020 | China | L | NLP/DL | Designed an NLP pipeline for the direct extraction of clinically relevant features of liver cancer from radiology reports | Retrospective study | Radiology reports |
| Mao et al.[26] | 2021 | China | L | ML | Investigate the performance of an ultrasound-based radiomics approach to differentiate primary liver cancer from metastatic liver cancer | Retrospective study | US images |
| Jang et al.[27] | 2021 | South Korea | G | DL/CV | Evaluate the diagnostic performance of AI in differentiating biliary lesions using EUS images | Retrospective study | EUS images |
| Dongyan et al.[28] | 2021 | China | G | DL/CV | Assessed duodenoscopy assisted by visual sensing technology based on convolutional neural network algorithm in the diagnosis and treatment of gallstones | Pilot study | ERCP/ surgery images |
| Kim et al.[29] | 2021 | South Korea | G | DL/CV | Aimed to differentiate gallbladder polyps in ultrasound images using deep learning | Retrospective study | US images |
| Yamashita et al.[30] | 2021 | USA | P | NLP | Identify patients with pancreatic cystic lesions and extract measurements from imaging reports using NLP | Retrospective study | Radiology reports |
| Chong et al.[31] | 2022 | China | L | CV/ML | Investigate the impact of MRI-based radiomics on predicting GPC3 expression and the relevant recurrence-free survival in liver cancer | Retrospective study | MRI images |
| Liu et al.[32] | 2022 | USA | L | ML | Machine learning-based methods to select clinical and morphologic features to differentiate hepatocellular adenoma subtypes | Retrospective study | Pathology specimens/patient records |
| Schuessler et al.[33] | 2022 | Germany | L | ML | Differentiation of hemodynamically significant and non-significant coronary stenoses in patients undergoing evaluation for liver transplant | Retrospective study | CTA images |
| Chang et al.[34] | 2022 | China | G | DL | Explore the application value of the neural network and genetic algorithms in the detection and prognosis of tumor markers in patients with gallbladder cancer | Retrospective study | Tumor-markers |
| Kooragayala | 2022 | USA | P | NLP | Utilized an NLP algorithm to quantify the incidence of clinically relevant pancreatic lesions in CT imaging | Retrospective study | Radiology reports |
Regarding sample size, most studies (n = 13) reporting diagnostic applications of AI in HPB surgery utilized data from fewer than 1,000 patients. The smallest number of patients in a focused study of the ultrasound-based classification of liver lesions was 22[14]. Three studies included over 5,000 patients and were included in our big data cohort[16,30,35]. The largest number of included patients was 199,783[30]. Most studies (n = 16) looking at prognostic uses of AI in HPB surgery had fewer than 500 patients. Two studies had fewer than 5,000 patients, but were included in our big data cohort due to the high number of images and image reports included[58,63]. Eleven studies looking at interventional uses of AI in HPB surgery did not use actual patient data, but used simulations-based approaches[69,70,75,76,81,82,84,85,91,92,104]. There was little mention or use of “explainable AI” concepts in any of the included studies.
Conceptual mapping of AI research in HPB surgery
Following data extraction and study classification, we undertook a conceptual mapping exercise to identify key areas and relationships in AI use [Figure 4]. Many of the identified concepts involved outcome prediction (such as the risk of complication, or personalized survival predictions). Others utilized AI to support clinicians in the identification of a condition before, during, or after surgery (such as identifying malignancy, identifying complications early, or even the prevention of these by using AI to alert clinicians to unseen structures intraoperatively). Preoperative planning and surgical simulation were particularly key areas within the intervention grouping. Finally, within the conceptual mapping exercise, we identified several areas where AI may be useful as either a risk stratification tool or as an intervention in future research (purple text, Figure 4).
Figure 4. Conceptual mapping of areas of AI research in HPB surgery, stratified by treatment timing. This exercise identified several areas of overlap (dashed arrows) across different divisions, in addition to several areas where AI would be useful for future research (purple free text). CT: Computed tomography; IPMN: intraductal papillary mucinous neoplasm; MRI: magnetic resonance imaging; PHLF: post hepatectomy liver failure; POPF: postoperative pancreatic fistula.
Diagnostic applications of artificial intelligence
Diagnostic applications of AI primarily involved interpreting images using computer vision models [Table 1 and Figure 4]. AI was used across a range of imaging modalities, including transabdominal ultrasound, endoscopic ultrasound, MRI and CT, to identify lesions or classify lesions into different radiomic subgroups of disease. Although the majority of preoperative, diagnostic AI work focused on imaging, there were studies investigating perioperative risk prediction. However, there were no studies that proposed to use AI as an intervention in preoperative care pathways. Therefore, it should be considered that preoperative AI may also be undertaken with a broader surgical focus, rather than specifically targeted at HPB populations and hence are not discussed in this review.
Prognostic applications of artificial intelligence
The majority of prognostic applications for AI were in the prediction of cancer recurrence and survival
Summary of included studies focusing on prognostic uses of AI in HPB surgery
| Authors | Year of publication | Location | Organ | AI method | Aim | Design | Data |
| Singal et al.[36] | 2013 | USA | L | ML | Develop and compare predictive models for HCC development among cirrhotic patients using conventional regression analysis and machine-learning algorithms | Prospective study | Patient factors |
| Banerjee et al.[37] | 2015 | USA | L | ML/CV | RVI was assessed for its ability to predict MVI and outcomes in patients with HCC who underwent surgical resection or liver transplant | Prospective evaluation of a retrospective cohort | CT images |
| Walczak et al.[38] | 2017 | USA | P | ML | Assess the accuracy of artificial neural networks in predicting survival in patients with pancreatic cancer using both clinical and patient-centered data | Retrospective study | Patient factors |
| Ying Zhou et al.[39] | 2017 | China | L | ML/CV | Develop a CT-based radiomics signature and assess its ability to preoperatively predict the early recurrence (≤ 1 year) of hepatocellular carcinoma (HCC) | Retrospective study | CT images |
| Zheng et al.[40] | 2018 | China | L | ML/CV | Developed a CT–based radiomic nomogram to predict recurrence-free survival rates for HCC after resection, ablation, and transplant | Retrospective study | CT images |
| Ivanics et al.[41] | 2019 | Canada | L | ML | Leverage machine learning to develop an accurate post-transplantation HCC recurrence prediction calculator | Retrospective study | Patient factors |
| Sala Elarre et al.[42] | 2019 | Spain | P | ML | Evaluated the 2-year relapse risk for pancreatic cancer patients based on a machine-learning algorithm | Retrospective study | Patient factors |
| Marinelli et al.[43] | 2019 | USA | L | NLP/DL | Determine if weakly supervised learning/active transfer learning can hasten clinical deployment of deep learning models for liver segmentation | Retrospective study | Radiology reports/CT images |
| Naseif et al.[44] | 2019 | USA | P | ML/CV | Develop a delta-radiomic process based on machine learning to predict the treatment response of pancreatic cancer | Retrospective study | CT images |
| Shan et al.[45] | 2019 | China | L | ML/CV | A Prediction model based on peritumoral radiomics signatures from CT - investigate its efficiency in predicting early recurrence of HCC after curative treatment | Retrospective study | CT images |
| Chen et al.[46] | 2020 | China | L | CV/ML | Establish a radiomics-based clinical model for preoperative prediction of PHLF in HCC | Retrospective study | MRI images |
| Han et al.[47] | 2020 | South Korea | P | ML | Risk prediction model for POPF using AI | Retrospective study | Patient factors |
| Kambakamba | 2020 | Switzerland | P | ML | The potential of machine learning-based approaches to describe the pancreatic texture and to predict POPF | Retrospective study | CT images |
| Merath et al.[49] | 2020 | USA | L/P | ML | Assess ML algorithm to predict the patient risk of developing complications following liver, pancreatic or colorectal surgery | Retrospective study | Patient factors |
| Saillard et al.[50] | 2020 | France | L | DL | Evaluate the effectiveness of AI algorithms to predict survival following HCC resection | Development and testing of AI models | Histology images |
| Cesaretti et al.[51] | 2020 | France Italy | L | ML/DL/CV | Automatizing liver-graft segmentation from smartphone images and validating the robustness of this approach | Prospective study | Surgery images |
| Mai et al.[52] | 2020 | China | L | DL | Establish and validate an artificial neural network model to predict severe post-hepatectomy liver failure in patients with hepatocellular carcinoma who underwent hemi-hepatectomy | Retrospective study | Patient factors |
| Liu et al.[53] | 2020 | Taiwan | L | ML | Devise a predictive model to predict postoperative survival within 30 days based on the patient’s preoperative physiological measurement values | Retrospective study | Patient factors |
| Schoenberg et al.[54] | 2020 | Germany | L | ML | Developing and validating a machine-learning algorithm to predict which patients are sufficiently treated by LR | Retrospective study | Patient factors |
| Szpakowski et al.[55] | 2020 | USA | G | NLP | Determine the growth pattern of GPs and their association with GBC | Retrospective study | Radiology reports |
| Capretti et al.[56] | 2021 | Italy Portugal | P | CV/ML | Develop a reliable and reproducible machine learning-based multimodal risk model capable of predicting CR-POPF by combining radiomic features and morphologic features | Retrospective study | CT images/patient factors |
| Sun et al.[57] | 2021 | China | L | DL | Develop a model to predict HCC recurrence | Retrospective study | Patient factors |
| Xie et al.[58] | 2021 | USA | P | NLP | Develop and apply a natural language processing algorithm for the characterization of patients diagnosed with chronic pancreatitis | Retrospective study | Radiology reports |
| Hayashi et al.[59] | 2022 | Japan | P | ML | Predict recurrence and metastatic sites in pancreatic cancer following curative surgery | Retrospective study | Histology images |
| Li et al.[60] | 2022 | China | P | ML | Develop and validate clinical-radiomics models that preoperatively predict 1 and 2-year recurrence of PDAC | Retrospective study | CT images/patient factors |
| Noh et al.[61] | 2022 | South Korea | L | ML | Machine learning-based survival rate prediction of hepatocellular carcinoma patients | Retrospective study | Patient factors |
| Morris-Stiff et al.[62] | 2022 | USA | G | NLP | Develop a clinical prediction model for asymptomatic gallstones | Retrospective study | Radiology reports |
| Narayan et al.[63] | 2022 | USA | L | ML/CV | Developed an objective, computer vision artificial intelligence (CVAI) platform to score donor liver steatosis and compared its capability for predicting EAD against pathologist steatosis scores | Retrospective study | Histology images |
| Cotter et al.[64] | 2022 | USA | G | ML | Machine-based learning approach to stratify patients with gallbladder cancer into distinct prognostic groups using preoperative variables | Retrospective study | Patient factors |
Interventional applications of artificial intelligence
We identified several key concepts around supporting interventions with AI assistance [Table 3 and Figure 4]. Intraoperative vision was a major area, with multiple studies focusing on improving the visualization of unseen structures, which may cause significant patient harm if inadvertently injured (e.g., major blood vessels or the bile duct). This was achieved through virtual or augmented reality, where inputs from other data sources such as CT and MRI are combined (sensor fusion) and overlain on real-time images (e.g., through laparoscopic/robot-assisted surgery video source) to produce an augmented view of the surgical field.
Summary of included studies focusing on interventional uses of AI in HPB surgery
| Author | Year of publication | Location | Organ | AI method | Aim | Design | Data |
| Spinczyk et al.[65] | 2012 | Poland | L | ML | Measurement of liver motion during surgery | Single center feasibility study | Surgery videos |
| Okamato et al.[66] | 2012 | Japan | L/G | CV | Evaluate the utility of an image display system for augmented reality in hepatobiliary surgery under laparotomy | Case series | CT images |
| Fang et al.[67] | 2013 | China | L | CV | Assess the use of 3d planning for hepatectomy for hepatolithiasis | Retrospective study | CT images |
| Zein et al.[68] | 2013 | USA | L | CV | Establish anatomical precision and volumetric accuracy in 3D-printed models for donors and recipients undergoing LDLT | Prospective paired case series | CT/MRI images |
| Shahin et al.[69] | 2014 | Germany | L | ML | Develop a navigation approach to quickly compensate for tumor movements due to surgical manipulation | Description of experiment | US images |
| Yang et al.[70] | 2014 | South Korea | L | ML | Develop a user-centered 3D virtual liver surgery planning system algorithm | Pilot study | CT images |
| Fang et al.[71] | 2014 | China | P | CV | Investigate the clinical significance of 3-dimensional reconstruction of peripancreatic vessels for patients with suspected pancreatic cancer | Randomized parallel single-blind study | CT images |
| Begin et al.[72] | 2014 | Canada | L | CV | Evaluate an alternative automatic technique of liver volumetry based on a novel 3D virtual planning software and compare it to the manual technique | Prospective study | CT images |
| Bliznakova et al.[73] | 2015 | Bulgaria | L | CV/ML | Develop and test a software application for evaluation of the residual function of the liver prior to the intervention of the surgeons | Case series | CT images |
| Katic et al.[74] | 2015 | Germany | P/G | DL | Demonstrate the usefulness of deep learning model to identify surgical steps during laparoscopic cholecystectomy and pancreatic resections | Case series | Surgery videos |
| Song et al.[75] | 2015 | UK | L | ML | Describe a freehand laparoscopic ultrasound-based system that registers liver vessels in ultrasound with MR/CT data | Description of experiment and case series | US/CT/MRI images |
| Wang et al.[76] | 2015 | China USA | L | ML | Demonstrate the potential of homotopy-based SSC for shape-prior modeling in the liver surgical planning system | Description of experiment | CT images |
| Fang et al.[77] | 2015 | China | L | CV | Compare outcomes of surgery on centrally located HCC with and without 3D planning | Retrospective study | CT/MRI images |
| Zhang et al.[78] | 2015 | China | G | CV | Assess the use of 3d planning in surgery on bile duct cancer | Case series | CT images |
| Okuda et al.[79] | 2015 | Japan | G | CV | Evaluate the impact of 3D CT cholangiography on operative planning and outcomes of biliary malignancies | Retrospective study | CT images |
| Okamato et al.[80] | 2015 | Japan | P | CV | Evaluate the utility of navigation surgery using augmented reality technology for pancreatectomy | Case series | CT images |
| Fortmeier et al.[81] | 2016 | Germany | G/L | CV | Creation of a visuo-haptic simulation framework for the training and planning of the first steps of PTCD | Description of experiment | X-ray/US/CT images |
| Fusaglia et al.[82] | 2016 | Switzerland | L | CV | Present a novel LRS-based IGS system for laparoscopic liver procedures | Description of experiment | Laparoscopic surgery images |
| Ntourakis et al.[83] | 2016 | France | L | CV | Investigate the potential of AR-based navigation to help locate and resect colorectal liver metastases | Prospective pilot study | CT/MRI images |
| Mastmeyer et al.[84] | 2017 | Germany | L | ML | Compare axial force errors of simulated needle insertion for liver biopsy | Description of experiment | US/CT images |
| Sauer et al.[85] | 2017 | Germany | L | CV | Evaluates the application of a mixed reality head-mounted display for the visualization of anatomical structures during liver surgery | Case study | CT images |
| Cai et al.[86] | 2017 | China | L | CV | Report experience of using a 3d visualization system during hepatic resection | Case series | CT images |
| Miyamoto et al.[87] | 2017 | Japan | P | CV | 3d planning - compared the pancreatic duct diameter and location with the intraoperative findings | Retrospective study | CT images |
| Hu et al.[88] | 2018 | China | L | CV | Assess the use of 3d planning for specific hepatectomy | Retrospective study | CT images |
| Mise et al.[89] | 2018 | Japan | L | CV | Assess how virtual hepatectomy conducted using surgical planning software influences the outcomes of liver surgery | Retrospective study | CT images |
| Mascagani et al.[90] | 2019 | Italy France | G | DL | Develop and test a method for consistent critical view of safety evaluation and reporting in videos which could be developed into the deep learning model | Pilot study | Laparoscopic surgery images |
| Teatini et al.[91] | 2019 | Norway | L | ML | Test if intraoperative imaging is necessary for accurate surgical navigation for laparoscopic liver resection | Description of experiment | CT images |
| Ho et al.[92] | 2020 | New Zealand | L | CV | Describe the computational pipeline that integrates into silico liver models and algorithms to aid surgical planning for liver resection | Description of experiment | CT/US/MRI images |
| Prevost et al.[93] | 2020 | Switzerland | L | CV | Evaluate the technical feasibility and the clinical impact of a new augmented reality system for laparoscopic liver surgery | Pilot study | CT/MRI images |
| Sandal et al.[94] | 2021 | Turkey | G | ML | Determine the usefulness of fuzzy logic algorithm to evaluate risk in patients undergoing laparoscopic cholecystectomy | Case series | Patient factors |
| Cervantes-Sanchez | 2021 | Mexico Germany | L/G | ML/DL | Machine/deep learning methods are combined with HSI-goal is the automatic discrimination using HSI of the bile duct from the gallbladder and liver | Description of experiment and case series | Hyperspectral images |
| Tokuyasu et al.[96] | 2021 | Japan | G | DL/CV | Develop a system that outlines laparoscopic cholecystectomy landmarks on endoscopic images in real time | Description of experiment and case report | Laparoscopic surgery images |
| Guzman-Garcia | 2021 | Spain | G | NLP/DL | Assess if analysis of surgeons’ speech using natural language processing provide deeper insight into the surgical decision-making processes during laparoscopic cholecystectomy | Description of experiment | Audio transcripts of surgical videos |
| Imler et al.[98] | 2021 | USA | G | NLP/ML | Demonstrate the feasibility of using NLP to measure adherence to ERCP quality indicators across individual providers | Retrospective study | ERCP procedure reports |
| Ruzzenente et al.[99] | 2022 | Italy | L | ML | Evaluate four difficulty scoring systems in liver surgery and determine the most important characteristics using random forest models | Case series | Patient factors |
| Mascagani et al.[100] | 2022 | France Italy | G | DL/CV | Creation of an assessment tool for CVS | Multicentre retrospective validation | Annotated surgery videos |
| Mascagani et al.[101] | 2022 | France Italy | G | DL/CV | Develop a deep learning model to automatically segment hepatocystic anatomy and assess the criteria defining the critical view of safety (CVS) | Case series | Annotated surgery images |
| Tranter-Entwistle | 2022 | New Zealand Australia | G | ML/CV | Use a commercially available ML-driven platform to evaluate a subjective grading of operative difficulty in laparoscopic cholecystectomy | Case series | Surgery videos |
| Liu et al.[103] | 2022 | China | G | ML/CV | Develop model and preliminarily verify its potential surgical guidance ability by comparing its performance with surgeons during laparoscopic cholecystectomy | Pilot study | Annotated surgery images |
| Ugail et al.[104] | 2022 | UK | L | ML/DL/CV | Present the use of deep learning for the non-invasive evaluation of donor liver organs | Pilot study | Surgical images |
| Mojtahed et al.[105] | 2022 | USA Netherlands Portugal | L | DL/CV | Demonstrate the accuracy and precision of liver segment volume measurements | Retrospective study | MRI images |
| Han et al.[106] | 2022 | China | L | DL/CV | Develop and validate a three-dimensional convolutional neural network model for automatic liver segment segmentation | Retrospective study | MRI images |
| Ward et al.[107] | 2022 | USA | G | DL/CV | Trained model to identify PGS | Development and testing of AI models | Annotated surgery images |
| Madani et al.[108] | 2022 | Canada USA UK | G | DL/CV | Develop and evaluate the performance of models that can identify safe and dangerous zones of dissection during laparoscopic cholecystectomy | Development and testing of AI models | Annotated surgery images |
| Loukas et al.[109] | 2022 | Greece | G | DL/CV | Framework for vascularity classification of the gallbladder wall from intraoperative images of laparoscopic cholecystectomy | Development and testing of AI models | Surgery images |
| Golany et al.[110] | 2022 | Israel | G | DL/CV | Developed algorithm and evaluated its performance in recognizing surgical phases of laparoscopic cholecystectomy | Development and testing of AI models | Annotated surgery videos |
Preoperative surgical planning and simulation were also identified as key concepts. There were numerous studies that aimed to develop virtual reality models or other digital interventions which permitted surgeons to plan complex operations with the aim of minimizing complications. This was proposed to be achieved through pre-surgery operative simulation/rehearsal (advantages when unusual anatomy identified) or by using AI methods to predict severe complications such as post-hepatectomy liver failure (PHLF).
Artificial intelligence tasks
We identified several common AI tasks being applied in HPB surgery. Classification is where data can be assigned to groups based on a defined shared characteristic. Classification algorithms were frequently derived from imaging to group lesions into disease subgroups[15,18,21]. In another example, decision tree models were used to predict the occurrence of any complication and of specific complications in patients undergoing liver, pancreatic and colorectal surgery[49]. These algorithms were superior to the American Society of Anaesthesiologists (ASA) classification at predicting the chance of any complication. They performed well for specific complications, with c-statistics ranging from 0.76 to 0.98. As described in our conceptual mapping exercise, the augmentation of surgical fields to highlight relevant anatomy is a key area of research. This is an example of object detection and is a task well suited to laparoscopic cholecystectomy. Madani et al. describe a deep learning algorithm that intraoperatively recognizes “go,” or “no-go” areas of dissection to minimize the risk of adverse events such as bile duct injury[108].
The intersection of AI and big data in HPB surgery
We identified eleven studies utilizing large datasets in HPB surgery applications [Table 4]. Nine have been published since 2020. Eight of the 11 identified papers utilized NLP to extract data from large numbers of reports, mainly with the aim of identifying patients with a specific condition, either for phenotyping or to identify patient cohorts. The majority originated from the USA (n = 9; 82%), with one study from China and one from South Korea [Figure 5].
Summary of studies leveraging large datasets for AI use in HPB surgery
| Author | Year of publication | Study description |
| Roch et al.[16] | 2015 | 566,233 CT reports from 50,669 patients analysed for keywords associated with Pancreatic Cysts using NLP |
| Yang et al.[19] | 2017 | The Cancer Genome Atlas catalogs genes associated with 33 cancers. Genes associated with HCC were extracted from database and checked for overlap with genes identified in 35 years of published literature using NLP |
| Merath | 2020 | 15,657 patients undergoing liver, pancreatic or colorectal surgery (685 liver and 6,012 pancreatic) retrospectively were identified from the American College of Surgeons National Surgical Quality Improvement Program database. Risk-prediction Machine Learning model created from pre-op characteristics |
| Szpakowski | 2020 | 365 Gallbladder Cancer and 35,970 Gallbladder Polyp patients were identified from 622,227 patients in a Californian health system. NLP was used to identify Polyps from Ultrasound reports |
| Xie et al.[58] | 2021 | 58,085 imaging reports from 6,346 Chronic Pancreatitis patients were used to develop an NLP algorithm that could characterize features of Chronic Pancreatitis |
| Yamashita | 2021 | 430,426 imaging reports from 199,783 patients were used to create an NLP algorithm to identify the presence and size of Pancreatic Cysts |
| Imler et al.[98] | 2021 | 23,674 ERCP reports were analyzed for quality measures using NLP |
| Noh et al.[61] | 2022 | Machine learning-based prediction models for survival applied to 10,742 HCC patients |
| Morris-Stiff | 2022 | Ultrasound reports identified 49,414 patients with gallstones. NLP algorithm trained to identify asymptomatic patients (22,257) |
| Narayan | 2022 | 25,494 images from 90 liver biopsies were used to develop Machine Learning Computer Vision models to score liver steatosis |
| Kooragayala | 2022 | NLP was used to identify pancreatic lesions from 18,769 adult trauma CT reports |
Figure 5. Number of studies utilizing large datasets stratified by AI approach.
An example of the use of an NLP algorithm to identify patient cohorts and devise a means of following-up incidental scan results was by Kooragayala et al.[35]. This study used a keyword search associated with suspicious pancreatic lesions in over 18,000 patients who underwent a CT scan following trauma over a 10-year period. The approach identified pancreatic lesions in the reports of 232 patients, of which 48 were intraductal papillary mucinous neoplasms (IPMNs). In addition, this paper proposed a management flowchart for incidentally found pancreatic lesions. A further example of the use of NLP in high-volume data was demonstrated by Morris-Stiff et al.[62], who used NLP to identify asymptomatic gallstones from a cohort of 49,414 patients. They were then able to identify risk factors for progression to symptomatic gallstone disease in this asymptomatic cohort and showed an approximately 2% risk of symptomatic progression per year.
DISCUSSION
This review has identified a rapid increase in the quantity of AI research conducted within HPB surgery. Much of this is focused on intraoperative applications of AI, such as the use of image analysis and computer vision to address diagnostic and prognostic uncertainties. In addition, the use of 3D reconstruction and augmented reality models coupled with data-driven prediction algorithms has emerged as an important area, particularly in preoperative planning and intraoperative decision-making in liver surgery. Artificial intelligence methods have the most to offer in the distillation of multi-dimensional information to tractable knowledge that can be applied to individual treatment decisions. HPB surgery represents a good target for these technologies, given the frequently complex disease patterns and diverse treatment pathways employed.
Most artificial intelligence approaches rely on large volumes of data for training purposes. Of the commonly described features of big data, the included studies reflect “volume” and “variety” with fewer utilizing real-time rapidly changing data (“velocity”). Data sources included large pre-existing databases, collated images and imaging reports. Two notable databases used were the Cancer Genome Atlas and the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) database, which were widely used across a range of studies. Natural language processing was frequently employed to extract information from imaging reports and other healthcare text sources. In one study, NLP was used to identify concerning pancreatic lesions in historical imaging reports[35]. This demonstrates the depth and flexibility in AI techniques to adapt to changes in patient management over time - the malignant potential of particular pancreatic cysts has only been appreciated in recent years. Moreover, these approaches may be adapted to help non-specialists managing HPB conditions, particularly in low-resource settings with limited access to tertiary HPB services. As computer vision approaches improve, the supplementation of local imaging and pathology reporting with AI-derived diagnostic support may leapfrog the requirement for massive and often unaffordable training of humans to perform these tasks.
There are, however, genuine risks of bias arising with the development of these techniques. We found significant geographical variation in current research, with no studies incorporating data from low- and middle-income countries (LMICs). If the benefits of AI are to be shared equitably across contexts, then investigators must consider how solutions can broadly generalize between populations and avoid exacerbating pre-existing healthcare disparities. This is a widely discussed and controversial topic in the broader AI field. Inherent systematic biases in datasets clearly exist, with some of the most obvious reflecting racial, socioeconomic and gender-based prejudices. Addressing these complex issues is crucial across all AI work, including in HPB surgery. The majority of HPB disease occurs LMICs[111], so it is essential that these populations are better represented in current HPB research more broadly and AI research specifically.
In addition to geographical disparities, concerns around the transparency of AI algorithms and lack of explainability are likely to hamper uptake and trust in clinical practice[112]. The need for explainability is rooted in evidence-based medicine, which relies on transparency and reproducibility in decision-making[113]. Without explainable AI, patient trust in healthcare will erode. Others have argued that true explainability represents a false hope, and that explainability methods cannot deliver meaningful patient-level interpretability[114]. The focus should be on robust internal and external validation. In this review, we found little reference to concepts of explainability in included studies. It is important that these issues are explored and addressed, particularly when developing algorithms orientated toward patient-facing prognostication. As AI systems transition from research to clinical practice, transparency and reliability are paramount if trust is to be built and maintained[115,116].
The ability to understand and reproduce scientific findings is imperative, yet reporting the quality of included studies was variable. A number of useful reporting guidelines now exist, specifically orientated toward AI. In 2019, a rigorous process of literature review, expert consultation, Delphi survey, and consensus meeting resulted in the SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials - Artificial Intelligence) and CONSORT-AI (Consolidated Standards of Reporting Trials - Artificial Intelligence) standards[117]. In addition, two additional tools are currently under development: Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis AI extension (TRIPOD-AI) and the Prediction model Risk Of Bias Assessment Tool (PROBAST-AI)[118]. These promise to provide standardization and assessment tools that will greatly increase the quality of clinically-orientated AI study reporting.
Where should AI research in HPB be focussed? Most studies in this review concentrated on image analysis. While this is an important area, there are many other challenges in HPB which could benefit from the application of AI. Research prioritization in AI must be determined by broad stakeholder groups, led primarily by the patient and public representatives, accounting for a range of viewpoints and actively engaging non-technical individuals in the design and delivery of research studies. We found little mention of engagement with stakeholder groups in included studies (e.g., patients, clinicians and the wider HPB community), which is crucial if these complex interventions are to move into clinical practice successfully. Moreover, included studies focused on the development of AI models rather than on the implementation of AI systems. While this is understandable given the current stage of development, future work should focus on how broader AI-driven systems can be implemented safely into clinical pathways and be clear about the function they serve.
Our study has several limitations. First, there is significant heterogeneity in the content and outcomes of the various studies included. While meaningful comparisons are challenging, a useful overview of common issues and themes affecting AI research in HPB is provided. Second, as is the nature of a scoping review, it is possible that studies meeting the inclusion criteria have been omitted, leading to an incomplete presentation of the current literature. For example, papers focusing on NLP and the gallbladder were relatively poorly represented in exploratory literature searches, possibly reflecting poor search descriptors and study labeling. Finally, as AI and associated concepts are undergoing rapid development, study inclusion criteria are in flux. Improving formal definitions in these emerging fields will help study classification and ease of literature identification.
The use of AI and big data in HPB surgery and medicine, more generally, is rapidly expanding. AI promises benefits in the delivery of clinical care and may result in future improvement of healthcare outcomes. This review identifies crucial interlinking conceptual areas of AI as applied to HPB surgery. Future research must address issues of bias, transparency, and explainability and ensure that innovation is representative of HPB patient populations across the world.
DECLARATIONS
Authors’ contributions
Participated in the design of the study, data collection, screening, interpretation and presentation, writing of the manuscript and submitted the manuscript: McGivern KG
Participated in the design of the study, data collection, screening, interpretation and presentation, and critical evaluation of the manuscript: Knight SR
Participated in the writing and critical evaluation of the manuscript: Drake TM
Participated in data screening and presentation: Lucocq J
Participated in the critical evaluation of the manuscript: Bernabeu MO, Clark N, Fairfield C, Pius R, Shaw C, Seth S
Participated in the design of the study and critical evaluation of the manuscript: Harrison EM
All authors approved the final version of the manuscript
Availability of data and materials
Not applicable.
Financial support and sponsorship
None.
Conflicts of interest
All authors declared that there are no conflicts of interest.
Ethical approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Copyright
© The Author(s) 2023.
REFERENCES
1. McCarthy J. What is artificial intelligence? Available from: https://www.diochnos.com/about/McCarthyWhatisAI.pdf [Last accessed on 23 Mar 2023].
2. Hashimoto DA, Rosman G, Rus D, Meireles OR. Artificial intelligence in surgery: promises and perils. Available from: https://journals.lww.com/annalsofsurgery/Abstract/2018/07000/Artificial_Intelligence_in_Surgery__Promises_and.13.aspx [Last accessed on 23 Mar 2023].
3. Gumbs AA, Alexander F, Karcz K, et al. White paper: definitions of artificial intelligence and autonomous actions in clinical surgery. Art Int Surg 2022;2:93-100.
4. Gumbs AA, Perretta S, d’Allemagne B, Chouillard E. What is Artificial Intelligence Surgery? Art Int Surg 2021;1:1-10.
5. Elyan E, Vuttipittayamongkol P, Johnston P, et al. Computer vision and machine learning for medical image analysis: recent advances, challenges, and way forward. Art Int Surg ;2022:2.
6. Bari H, Wadhwani S, Dasari BVM. Role of artificial intelligence in hepatobiliary and pancreatic surgery. World J Gastrointest Surg 2021;13:7-18.
8. Vedula SS, Hager GD. Surgical data science: The new knowledge domain. Innov Surg Sci 2017;2:109-21.
9. NHS England. 2022/23 priorities and operational planning guidance. Available from: https://www.england.nhs.uk/wp-content/uploads/2022/02/20211223-B1160-2022-23-priorities-and-operational-planning-guidance-v3.2.pdf [Last accessed on 23 Mar 2023].
10. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018;169:467-73.
11. Knight SR, Ots R, Maimbo M, Drake TM, Fairfield CJ, Harrison EM. Systematic review of the use of big data to improve surgery in low- and middle-income countries. Br J Surg 2019;106:e62-72.
12. Covidence. Veritas health innovation, Melbourne, Australia. Available from: https://www.covidence.org/ [Last accessed on 23 Mar 2023].
13. Săftoiu A, Vilmann P, Gorunescu F, et al. Efficacy of an artificial neural network-based approach to endoscopic ultrasound elastography in diagnosis of focal pancreatic masses. Clin Gastroenterol Hepatol 2012;10:84-90.e1.
14. Wu K, Chen X, Ding M. Deep learning based classification of focal liver lesions with contrast-enhanced ultrasound. Optik 2014;125:4057-63.
15. Gatos I, Tsantis S, Karamesini M, Skouroliakou A, Kagadis G. Development of a support vector machine - based image analysis system for focal liver lesions classification in magnetic resonance images. J Phys Conf Ser 2015;633:012116.
16. Roch AM, Mehrabi S, Krishnan A, et al. Automated pancreatic cyst screening using natural language processing: a new tool in the early detection of pancreatic cancer. HPB 2015;17:447-53.
17. Sada Y, Hou J, Richardson P, El-Serag H, Davila J. Validation of case finding algorithms for hepatocellular cancer from administrative data and electronic health records using natural language processing. Med Care 2016;54:e9-14.
18. Kondo S, Takagi K, Nishida M, et al. Computer-aided diagnosis of focal liver lesions using contrast-enhanced ultrasonography with perflubutane microbubbles. IEEE Trans Med Imaging 2017;36:1427-37.
19. Yang H, Zhang X, Cai XY, et al. From big data to diagnosis and prognosis: gene expression signatures in liver hepatocellular carcinoma. PeerJ 2017;5:e3089.
20. Kuwahara T, Hara K, Mizuno N, et al. Usefulness of deep learning analysis for the diagnosis of malignancy in intraductal papillary mucinous neoplasms of the pancreas. Clin Transl Gastroenterol 2019;10:1-8.
21. Shen X, Yang F, Yang P, et al. Non-invasive diagnosis model for pancreatic cystic tumors based on machine learning-radiomics using contrast-enhanced CT. Available from: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3294088 [Last accessed on 27 Mar 2023].
22. Xu L, Yang P, Liang W, et al. A radiomics approach based on support vector machine using MR images for preoperative lymph node status evaluation in intrahepatic cholangiocarcinoma. Theranostics 2019;9:5374-85.
23. Brown AD, Kachura JR. Natural language processing of radiology reports in patients with hepatocellular carcinoma to predict radiology resource utilization. J Am Coll Radiol 2019;16:840-4.
24. Watson MD, Lyman WB, Passeri MJ, et al. Use of artificial intelligence deep learning to determine the malignant potential of pancreatic cystic neoplasms with preoperative computed tomography imaging. Am Surg 2021;87:602-7.
25. Liu H, Xu Y, Zhang Z, et al. A natural language processing pipeline of chinese free-text radiology reports for liver cancer diagnosis. IEEE Access 2020;8:159110-9.
26. Mao B, Ma J, Duan S, Xia Y, Tao Y, Zhang L. Preoperative classification of primary and metastatic liver cancer via machine learning-based ultrasound radiomics. Eur Radiol 2021;31:4576-86.
27. Jang SI, Kim YJ, Kim EJ, et al. Diagnostic performance of endoscopic ultrasound-artificial intelligence using deep learning analysis of gallbladder polypoid lesions. J Gastroenterol Hepatol 2021;36:3548-55.
28. Li D, Du B, Shen Y, Ge L, Lv H. Artificial intelligence-assisted visual sensing technology under duodenoscopy of gallbladder stones. J Sensors 2021;2021:1-13.
29. Kim T, Choi YH, Choi JH, Lee SH, Lee S, Lee IS. Gallbladder polyp classification in ultrasound images using an ensemble convolutional neural network model. J Clin Med 2021;10:3585.
30. Yamashita R, Bird K, Cheung PY, et al. Automated identification and measurement extraction of pancreatic cystic lesions from free-text radiology reports using natural language processing. Radiol Artif Intell 2022;4:e210092.
31. Chong H, Gong Y, Zhang Y, Dai Y, Sheng R, Zeng M. Radiomics on gadoxetate disodium-enhanced mri: non-invasively identifying glypican 3-positive hepatocellular carcinoma and postoperative recurrence. Acad Radiol 2023;30:49-63.
32. Liu Y, Liu YZ, Sun L, Zen Y, Inomoto C, Yeh MM. Subtyping of hepatocellular adenoma: a machine learning-based approach. Virchows Arch 2022;481:49-61.
33. Schuessler M, Saner F, Al-Rashid F, Schlosser T. Diagnostic accuracy of coronary computed tomography angiography-derived fractional flow reserve (CT-FFR) in patients before liver transplantation using CT-FFR machine learning algorithm. Eur Radiol 2022;32:8761-8.
34. Chang Y, Wu Q, Chi L, Huo H, Li Q. Adoption of combined detection technology of tumor markers via deep learning algorithm in diagnosis and prognosis of gallbladder carcinoma. J Supercomput 2022;78:3955-75.
35. Kooragayala K, Crudeli C, Kalola A, et al. Utilization of natural language processing software to identify worrisome pancreatic lesions. Ann Surg Oncol 2022;29:8513-9.
36. Singal AG, Mukherjee A, Elmunzer BJ, et al. Machine learning algorithms outperform conventional regression models in predicting development of hepatocellular carcinoma. Am J Gastroenterol 2013;108:1723-30.
37. Banerjee S, Wang DS, Kim HJ, et al. A computed tomography radiogenomic biomarker predicts microvascular invasion and clinical outcomes in hepatocellular carcinoma. Hepatology 2015;62:792-800.
38. Walczak S, Velanovich V. An evaluation of artificial neural networks in predicting pancreatic cancer survival. J Gastrointest Surg 2017;21:1606-12.
39. Zhou Y, He L, Huang Y, et al. CT-based radiomics signature: a potential biomarker for preoperative prediction of early recurrence in hepatocellular carcinoma. Abdom Radiol 2017;42:1695-704.
40. Zheng BH, Liu LZ, Zhang ZZ, et al. Radiomics score: a potential prognostic imaging feature for postoperative survival of solitary HCC patients. BMC Cancer 2018;18:1148.
41. Ivanics T, Nelson W, Patel MS, et al. The toronto postliver transplantation hepatocellular carcinoma recurrence calculator: a machine learning approach. Liver Transpl 2022;28:593-602.
42. Sala Elarre P, Oyaga-Iriarte E, Yu KH, et al. Use of machine-learning algorithms in intensified preoperative therapy of pancreatic cancer to predict individual risk of relapse. Cancers 2019;11:606.
43. Marinelli B, Kang M, Martini M, et al. Combination of active transfer learning and natural language processing to improve liver volumetry using surrogate metrics with deep learning. Radiol Artif Intell 2019;1:e180019.
44. Nasief H, Zheng C, Schott D, et al. A machine learning based delta-radiomics process for early prediction of treatment response of pancreatic cancer. NPJ Precis Oncol 2019;3:25.
45. Shan QY, Hu HT, Feng ST, et al. CT-based peritumoral radiomics signatures to predict early recurrence in hepatocellular carcinoma after curative tumor resection or ablation. Cancer Imaging 2019;19:11.
46. Chen Y, Liu Z, Mo Y, et al. Prediction of post-hepatectomy liver failure in patients with hepatocellular carcinoma based on radiomics using Gd-EOB-DTPA-enhanced MRI: the liver failure model. Front Oncol 2021;11:605296.
47. Han IW, Cho K, Ryu Y, et al. Risk prediction platform for pancreatic fistula after pancreatoduodenectomy using artificial intelligence. World J Gastroenterol 2020;26:4453-64.
48. Kambakamba P, Mannil M, Herrera P, et al. Machine learning based texture analysis predicts postoperative pancreatic fistula in preoperative non-contrast enhanced computed tomography. HPB 2020;22:S384.
49. Merath K, Hyer JM, Mehta R, et al. Use of machine learning for prediction of patient risk of postoperative complications after liver, pancreatic, and colorectal surgery. J Gastrointest Surg 2020;24:1843-51.
50. Saillard C, Schmauch B, Laifa O, et al. Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatology 2020;72:2000-13.
51. Cesaretti M, Brustia R, Goumard C, et al. Use of artificial intelligence as an innovative method for liver graft macrosteatosis assessment. Liver Transpl 2020;26:1224-32.
52. Mai RY, Lu HZ, Bai T, et al. Artificial neural network model for preoperative prediction of severe liver failure after hemihepatectomy in patients with hepatocellular carcinoma. Surgery 2020;168:643-52.
53. Liu CL, Soong RS, Lee WC, Jiang GW, Lin YC. Predicting short-term survival after liver transplantation using machine learning. Sci Rep 2020;10:5654.
54. Schoenberg MB, Bucher JN, Koch D, et al. A novel machine learning algorithm to predict disease free survival after resection of hepatocellular carcinoma. Ann Transl Med 2020;8:434.
55. Szpakowski JL, Tucker LY. Outcomes of gallbladder polyps and their association with gallbladder cancer in a 20-year cohort. JAMA Netw Open 2020;3:e205143.
56. Capretti G, Bonifacio C, De Palma C, et al. A machine learning risk model based on preoperative computed tomography scan to predict postoperative outcomes after pancreatoduodenectomy. Updates Surg 2022;74:235-43.
57. Sun LY, Ouyang Q, Cen WJ, Wang F, Tang WT, Shao JY. A model based on artificial intelligence algorithm for monitoring recurrence of HCC after hepatectomy. Am Surg 2021;11:31348211063549.
58. Xie F, Chen Q, Zhou Y, et al. Characterization of patients with advanced chronic pancreatitis using natural language processing of radiology reports. PLoS One 2020;15:e0236817.
59. Hayashi K, Ono Y, Takamatsu M, et al. Prediction of recurrence pattern of pancreatic cancer post-pancreatic surgery using histology-based supervised machine learning algorithms: a single-center retrospective study. Ann Surg Oncol ;2022:4624-34.
60. Li X, Wan Y, Lou J, et al. Preoperative recurrence prediction in pancreatic ductal adenocarcinoma after radical resection using radiomics of diagnostic computed tomography. EClinicalMedicine 2022;43:101215.
61. Noh B, Park YM, Kwon Y, et al. Machine learning-based survival rate prediction of Korean hepatocellular carcinoma patients using multi-center data. BMC Gastroenterol 2022;22:85.
62. Morris-Stiff G, Sarvepalli S, Hu B, et al. The natural history of asymptomatic gallstones: a longitudinal study and prediction model. Clin Gastroenterol Hepatol 2023;21:319-327.e4.
63. Narayan RR, Abadilla N, Yang L, et al. Artificial intelligence for prediction of donor liver allograft steatosis and early post-transplantation graft failure. HPB 2022;24:764-71.
64. Cotter G, Beal EW, Poultsides GA, et al. Using machine learning to preoperatively stratify prognosis among patients with gallbladder cancer: a multi-institutional analysis. HPB 2022;24:1980-8.
65. Spinczyk D, Karwan A, Rudnicki J, Wróblewski T. Stereoscopic liver surface reconstruction. Wideochir Inne Tech Maloinwazyjne 2012;7:181-7.
66. Okamoto T, Onda S, Matsumoto M, et al. Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 2013;20:249-53.
67. Fang CH, Liu J, Fan YF, Yang J, Xiang N, Zeng N. Outcomes of hepatectomy for hepatolithiasis based on 3-dimensional reconstruction technique. J Am Coll Surg 2013;217:280-8.
68. Zein NN, Hanouneh IA, Bishop PD, et al. Three-dimensional print of a liver for preoperative planning in living donor liver transplantation. Liver Transpl 2013;19:1304-10.
69. Shahin O, Beširević A, Kleemann M, Schlaefer A. Ultrasound-based tumor movement compensation during navigated laparoscopic liver interventions. Surg Endosc 2014;28:1734-41.
70. Yang X, Yu HC, Choi Y, et al. Development and usability testing of Dr. LiverTM: a user-centered 3D virtual liver surgery planning system. HFES 2014;58:698-702.
71. Fang CH, Kong D, Wang X, et al. Three-dimensional reconstruction of the peripancreatic vascular system based on computed tomographic angiography images and its clinical application in the surgical management of pancreatic tumors. Pancreas 2014;43:389-95.
72. Bégin A, Martel G, Lapointe R, et al. Accuracy of preoperative automatic measurement of the liver volume by CT-scan combined to a 3D virtual surgical planning software (3DVSP). Surg Endosc 2014;28:3408-12.
73. Bliznakova K, Kolev N, Buliev I, et al. Computer aided preoperative evaluation of the residual liver volume using computed tomography images. J Digit Imaging 2015;28:231-9.
74. Katić D, Julliard C, Wekerle AL, et al. LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J Comput Assist Radiol Surg 2015;10:1427-34.
75. Song Y, Totz J, Thompson S, et al. Locally rigid, vessel-based registration for laparoscopic liver surgery. Int J Comput Assist Radiol Surg 2015;10:1951-61.
76. Wang G, Zhang S, Xie H, Metaxas DN, Gu L. A homotopy-based sparse representation for fast and accurate shape prior modeling in liver surgical planning. Med Image Anal 2015;19:176-86.
77. Fang CH, Tao HS, Yang J, et al. Impact of three-dimensional reconstruction technique in the operation planning of centrally located hepatocellular carcinoma. J Am Coll Surg 2015;220:28-37.
78. Zhang J, Qiao QL, Guo XC, Zhao JX. Application of three-dimensional visualization technique in preoperative planning of progressive hilar cholangiocarcinoma. Am J Transl Res 2018;10:1730-5.
79. Okuda Y, Taura K, Seo S, et al. Usefulness of operative planning based on 3-dimensional CT cholangiography for biliary malignancies. Surgery 2015;158:1261-71.
80. Okamoto T, Onda S, Yasuda J, Yanaga K, Suzuki N, Hattori A. Navigation surgery using an augmented reality for pancreatectomy. Dig Surg 2015;32:117-23.
81. Fortmeier D, Mastmeyer A, Schröder J, Handels H. A virtual reality system for PTCD simulation using direct visuo-haptic rendering of partially segmented image data. IEEE J Biomed Health Inform 2016;20:355-66.
82. Fusaglia M, Hess H, Schwalbe M, et al. A clinically applicable laser-based image-guided system for laparoscopic liver procedures. Int J Comput Assist Radiol Surg 2016;11:1499-513.
83. Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P. Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg 2016;40:419-26.
84. Mastmeyer A, Fortmeier D, Handels H. Evaluation of direct haptic 4D volume rendering of partially segmented data for liver puncture simulation. Sci Rep 2017;7:671.
85. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg 2017;266:706-12.
86. Cai W, Fan Y, Hu H, Xiang N, Fang C, Jia F. Postoperative liver volume was accurately predicted by a medical image three dimensional visualization system in hepatectomy for liver cancer. Surg Oncol 2017;26:188-94.
87. Miyamoto R, Oshiro Y, Nakayama K, et al. Three-dimensional simulation of pancreatic surgery showing the size and location of the main pancreatic duct. Surg Today 2017;47:357-64.
88. Hu M, Hu H, Cai W, et al. The safety and feasibility of three-dimensional visualization technology assisted right posterior lobe allied with part of V and VIII sectionectomy for right hepatic malignancy therapy. J Laparoendosc Adv Surg Tech A 2018;28:586-94.
89. Mise Y, Hasegawa K, Satou S, et al. How has virtual hepatectomy changed the practice of liver surgery? Ann Surg 2018;268:127-33.
90. Mascagni P, Fiorillo C, Urade T, et al. Formalizing video documentation of the critical view of safety in laparoscopic cholecystectomy: a step towards artificial intelligence assistance to improve surgical safety. Surg Endosc 2020;34:2709-14.
91. Teatini A, Pelanis E, Aghayan DL, et al. The effect of intraoperative imaging on surgical navigation for laparoscopic liver resection surgery. Sci Rep 2019;9:18687.
92. Ho H, Yu HB, Bartlett A, Hunter P. An in silico pipeline for subject-specific hemodynamics analysis in liver surgery planning. Comput Methods Biomech Biomed Engin 2020;23:138-42.
93. Prevost GA, Eigl B, Paolucci I, et al. Efficiency, accuracy and clinical applicability of a new image-guided surgery system in 3D laparoscopic liver surgery. J Gastrointest Surg 2020;24:2251-8.
94. Sandal B, Hacioglu Y, Salihoglu Z, Yagiz N. Fuzzy logic preanesthetic risk evaluation of laparoscopic cholecystectomy operations. Am Surg 2023;89:414-23.
95. Cervantes-sanchez F, Maktabi M, Köhler H, et al. Automatic tissue segmentation of hyperspectral images in liver and head neck surgeries using machine learning. Art Int Surg 2021;1:22-37.
96. Tokuyasu T, Iwashita Y, Matsunobu Y, et al. Development of an artificial intelligence system using deep learning to indicate anatomical landmarks during laparoscopic cholecystectomy. Surg Endosc 2021;35:1651-8.
97. Guzmán-García C, Gómez-Tome M, Sánchez-González P, Oropesa I, Gómez EJ. Speech-based surgical phase recognition for non-intrusive surgical skills' assessment in educational contexts. Sensors 2021;21:1330.
98. Imler TD, Sherman S, Imperiale TF, et al. Provider-specific quality measurement for ERCP using natural language processing. Gastrointest Endosc 2018;87:164-173.e2.
99. Ruzzenente A, Bagante F, Poletto E, et al. A machine learning analysis of difficulty scoring systems for laparoscopic liver surgery. Surg Endosc 2022;36:8869-80.
100. Mascagni P, Alapatt D, Laracca GG, et al. Multicentric validation of EndoDigest: a computer vision platform for video documentation of the critical view of safety in laparoscopic cholecystectomy. Surg Endosc 2022;36:8379-86.
101. Mascagni P, Vardazaryan A, Alapatt D, et al. Artificial intelligence for surgical safety: automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg 2022;275:955-61.
102. Tranter-entwistle I, Eglinton T, Connor S, Hugh TJ. Operative difficulty in laparoscopic cholecystectomy: considering the role of machine learning platforms in clinical practice. Art Int Surg 2022;2:46-56.
103. Liu R, An J, Wang Z, et al. Artificial intelligence in laparoscopic cholecystectomy: does computer vision outperform human vision? Art Int Surg 2022;2:80-92.
104. Ugail H, Abubakar A, Elmahmudi A, Wilson C, Thomson B. The use of pre-trained deep learning models for the photographic assessment of donor livers for transplantation. Art Int Surg 2022;2:101-19.
105. Mojtahed A, Núñez L, Connell J, et al. Repeatability and reproducibility of deep-learning-based liver volume and Couinaud segment volume measurement tool. Abdom Radiol 2022;47:143-51.
106. Han X, Wu X, Wang S, et al. Automated segmentation of liver segment on portal venous phase MR images using a 3D convolutional neural network. Insights Imaging 2022;13:26.
107. Ward TM, Hashimoto DA, Ban Y, Rosman G, Meireles OR. Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation. Surg Endosc 2022;36:6832-40.
108. Madani A, Namazi B, Altieri MS, et al. Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy. Ann Surg 2022;276:363-9.
109. Loukas C, Gazis A, Schizas D. Multiple instance convolutional neural network for gallbladder assessment from laparoscopic images. Int J Med Robot 2022;18:e2445.
110. Golany T, Aides A, Freedman D, et al. Artificial intelligence for phase recognition in complex laparoscopic cholecystectomy. Surg Endosc 2022;36:9215-23.
111. 2019 Diseases and Injuries Collaborators. Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet 2020;396:1204-22.
113. Reddy S. Explainability and artificial intelligence in medicine. Lancet Digit Health 2022;4:e214-5.
114. Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health 2021;3:e745-50.
115. Topol EJ. High-performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44-56.
116. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K. The practical implementation of artificial intelligence technologies in medicine. Nat Med 2019;25:30-6.
117. Rivera SC, Liu X, Chan AW, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Lancet Digit Health 2020;2:e549-60.
Cite This Article
Open Access
How to Cite
McGivern, K. G.; Drake, T. M.; Knight, S. R.; Lucocq, J.; Bernabeu, M. O.; Clark, N.; Fairfield, C.; Pius, R.; Shaw, C. A.; Seth, S.; Harrison, E. M. Applying artificial intelligence to big data in hepatopancreatic and biliary surgery: a scoping review. Art. Int. Surg. 2023, 3, 27-47. http://dx.doi.org/10.20517/ais.2022.39
Download Citation
Export Citation File:
Type of Import
Tips on Downloading Citation
Citation Manager File Format
Type of Import
Direct Import: When the Direct Import option is selected (the default state), a dialogue box will give you the option to Save or Open the downloaded citation data. Choosing Open will either launch your citation manager or give you a choice of applications with which to use the metadata. The Save option saves the file locally for later use.
Indirect Import: When the Indirect Import option is selected, the metadata is displayed and may be copied and pasted as needed.
About This Article
Special Issue
Copyright
Comments
Comments must be written in English. Spam, offensive content, impersonation, and private information will not be permitted. If any comment is reported and identified as inappropriate content by OAE staff, the comment will be removed without notice. If you have any queries or need any help, please contact us at [email protected].