The surgeon behind the algorithm: Why Intraoperative AI Literacy is a patient safety imperative

INTRODUCTION AND BACKGROUND

Artificial intelligence (AI) is entering the surgical operating room. AI-assisted guidance systems are increasingly being developed and evaluated to support anatomical identification, document safety-critical steps, and provide real-time intraoperative decision support^[1-5]. A 2025 international survey of surgeons reported growing awareness of AI, participation in AI-related training, and a strong willingness to integrate AI into surgical practice despite persistent gaps in formal knowledge and education^[6]. What has not advanced at the same pace is preparation for using these systems safely in practice.

Intraoperative decision-making in surgery depends on the surgeon’s ability to interpret anatomy under conditions of uncertainty. In procedures such as laparoscopic cholecystectomy, where critical structures may be obscured by inflammation, fibrosis, or anatomical variation, the consequences of misidentification can be severe and irreversible.

AI-assisted guidance systems introduce a new variable into this process. By presenting anatomical overlays, structural labels, and confidence scores in real time, these systems offer interpretive outputs that may influence surgical decision-making at moments when independent judgment is most critical. The central safety question is therefore not only whether AI can identify anatomy accurately, but also whether surgeons are prepared to evaluate its guidance, recognise its limits, and reject it when clinical judgment requires.

To the authors’ knowledge, no established competency framework currently defines the competencies needed by surgeons to use intraoperative AI guidance safely in clinical practice^[7,8]. Drawing on evidence from surgical AI research and the human factors literature, this paper proposes Intraoperative AI Literacy (IAIL) as a candidate framework for organising these competencies and guiding their future training and assessment.

Intraoperative AI guidance is not yet part of routine surgical practice. The systems discussed in this paper remain largely at the stage of technical development, feasibility testing, and early clinical validation^[4,9,10]. As such, the framework proposed here is anticipatory rather than descriptive. It is not a response to an established pattern of harm from routine clinical use; rather, it identifies competency gaps that may arise as intraoperative AI systems move towards routine clinical use. Others may reasonably propose different competencies or conclude that more is required than the five domains proposed here.

THE PROBLEM WITH TRUSTING THE SCREEN

Surgeons have always used tools that provide information. A cholangiogram, a fluoroscopic image, or a frozen section may guide intraoperative decision-making, but none of these tools determines the operative decision. They provide findings that the surgeon must interpret in context.

Intraoperative AI guidance, however, will differ from these well-established information tools in two important ways. First, AI may present its output as an already interpreted judgment. An overlay, label, or confidence score can create the appearance of certainty, not only for the surgeon but also for the wider operating team. Second, surgeons may have limited knowledge of how the AI system reached its conclusion: what cases it was trained on, which patient populations were represented in its training data, and where its reliability may degrade. Using AI safely will therefore require surgeons to judge when its output should be trusted and when it should not.

The most mature work in AI-assisted laparoscopic cholecystectomy comes from Mascagni et al.^[1,2,4,5], together with independent work from several other groups^[11-16].

The initial development study of the EndoDigest platform demonstrated that automatically extracted video clips successfully documented the critical view of safety (CVS) in 91% of laparoscopic cholecystectomy cases^[1]. In the subsequent multicentre validation study, this proportion decreased to 75% when the platform was evaluated across four independent centres, reflecting substantial variation in surgical workflows^[2].

In a separate retrospective analysis of 11 laparoscopic cholecystectomy videos containing bile duct injuries, 45% of the injury-causing interactions occurred outside both the AI-defined Go and No-Go zones^[17].

Madani et al. developed GoNoGoNet to flag safe and dangerous zones of dissection, finding that it changed surgeon decisions in more than one in four cases, mostly in a safer direction^[12,13]. Ward et al. trained AI to assess gallbladder inflammation from the opening laparoscopic view and predict operative difficulty before the first cut^[14]. Golany et al. demonstrated surgical phase recognition with 89% accuracy overall, with performance declining to 81% in the most complex cases^[15].

Across studies, countries, and technical approaches, the same pattern emerges: AI performs best when an operation is relatively straightforward and becomes less dependable as anatomy is distorted, inflammation becomes severe, or procedural complexity increases. These are precisely the circumstances in which surgeons most need reliable guidance and have the strongest reason to verify it independently.

This pattern is not confined to hepatobiliary surgery. In urology, a convolutional neural network has been applied to intraoperative video during robot-assisted radical prostatectomy to segment the bladder, prostate, and seminal vesicle-vas deferens in real time^[18]. The authors note, however, that the clinical utility of such segmentation during live surgery remains largely unknown^[18]. In colorectal surgery, a deep learning model has been developed to highlight areolar tissue as a landmark for the total mesorectal excision plane^[19]. The model was derived from a single-centre feasibility study involving 32 patients, and the authors identify the limited quantity and quality of training data as constraints on generalisability^[19]. Both systems therefore remain at an early stage of clinical evaluation.

These studies share a common limitation. Many current surgical AI models are developed from datasets assembled at one or a small number of centres for a single procedure, and therefore reflect the patient populations, operative techniques, and imaging conditions of those institutions^[10,18,19]. Performance demonstrated in these settings does not necessarily establish performance elsewhere. A surgeon treating a population that differs from the population on which an AI model was trained may therefore receive systematically less reliable guidance. Yet the system itself may provide no clear indication that its performance has degraded.

The safety problem, then, is not simply whether AI can identify anatomy accurately under ideal conditions. It is whether surgeons are trained to recognise when a case falls outside the conditions in which a system was developed and validated, to interpret its output with appropriate caution, and, when necessary, to discount or reject its guidance.

SURGERY HAS BEEN HERE BEFORE

This is not the first time surgery has faced a technological transition that changed how surgeons see anatomy, interpret information, and make decisions in the operating room.

When laparoscopic cholecystectomy was introduced in the late 1980s and early 1990s, bile duct injuries increased compared with open surgery^[20]. The problem was not simply one of technical skill. The operative view had changed. Anatomy appeared on a two-dimensional screen, spatial orientation became less intuitive, and surgeons were working in a visual environment for which their training had not prepared them. The most common cause of injury was misidentification of one structure as another. The surgical response was the CVS, introduced by Strasberg in 1995^[20]. Rather than relying on impression or confidence, it provided surgeons with a structured check before cutting: a way of verifying what they were seeing before acting on it.

A similar transition occurred with robotic surgery. Formal training pathways, simulation programmes, and credentialing systems gradually developed to support this transition^[21].

The World Health Organization (WHO) Surgical Safety Checklist reinforced the same principle more broadly: a simple structured verification step was associated with reductions in mortality and postoperative complications across diverse hospital settings^[22].

The pattern is familiar. A transformative technology enters the operating room, new demands on surgeon judgment emerge that the existing training infrastructure did not anticipate, and the field eventually develops formal training standards, structured safety frameworks, and oversight mechanisms to meet them. The training response may lag behind the technology, and in some cases the field catches up only after preventable harm has occurred.

AI-assisted guidance systems may represent the next iteration of this pattern.

LESSONS FROM HIGH-STAKES AUTOMATION

Across multiple safety-critical domains, the human factors literature has consistently shown that automation changes how people make decisions and supervise complex systems. As reliance on automation increases, maintaining independent judgment, recognising system limitations, and knowing when to intervene become increasingly important^[23-25].

Similar challenges have been recognised in fields such as aviation and nuclear power, where operators are required to supervise automated systems while remaining prepared to question or override them when necessary. Safe integration also requires training, clear operational procedures, and continued human oversight^[23-25].

Aviation provides the most extensively documented example of how a high-stakes field recognised these risks and developed a systematic response. As cockpit automation matured, accident investigations repeatedly identified failures in the interaction between pilots and automated systems, including overreliance on automation, loss of situational awareness, and difficulty transitioning to manual control when automation behaved unexpectedly or could no longer manage the situation. The Air France Flight 447 and Boeing 737 MAX accidents illustrate some of the complex human factors associated with highly automated systems^[26,27]. Aviation’s response was not to limit automation but to develop structured frameworks for its safe management, including regulatory oversight, simulator-based training, recurrent competency assessment, and explicit training in intervention and manual control.

Surgeons using intraoperative AI guidance systems will increasingly operate within a comparable human-machine environment. Frameworks for automation awareness, failure recognition, trust calibration, and override decision-making will therefore be needed. The risk is not simply that AI may produce unreliable guidance, but that the surgeon may not recognise it quickly enough to intervene safely^[23-25]. This suggests that safe integration of AI-assisted surgery will depend not only on technological performance but also on the surgeon’s ability to supervise, interpret, question, and override AI guidance, when necessary, in the operating room.

IAIL: TRAINING FOR THE TRANSITION

This paper proposes IAIL as a candidate competency framework comprising five core abilities: to interpret AI guidance in the operating room, evaluate its applicability to the operative context, act on it when appropriate, override it when clinical judgment indicates otherwise, and retain accountability for all operative decisions. The five domains that operationalise these abilities are defined in Table 1 and positioned at the bottom of Figure 1. IAIL has not undergone expert consensus, formal validity assessment, or clinical evaluation. It is therefore presented as a framework for testing and refinement rather than for adoption. Supplementary Figure 1 illustrates how these competencies may be applied when AI guidance conflicts with the surgeon’s direct operative assessment.

Figure 1. Human-AI co-adaptation in AI-assisted surgery: two conceptual patient-safety trajectories. The figure illustrates how increasing AI capability may follow either a safer or less safe trajectory depending on how surgeons and the surrounding surgical system adapt to its use. These trajectories are presented as a conceptual framework rather than empirically derived categories. The central Human-AI Surgical System includes workflow, governance, team, and communication elements that influence how AI guidance is integrated into operative care. The five domains of IAIL are shown at the base of the figure as the surgeon-level competency foundation proposed in this paper. While workflow, governance, and institutional factors contribute to safe adoption, the primary focus of this manuscript is the surgeon competencies required to interpret, verify, question, and override AI guidance appropriately. AI: Artificial intelligence; IAIL: Intraoperative AI Literacy.

Although the safe adoption of AI-assisted surgery will depend on factors beyond surgeon competency, including workflow, governance, institutional readiness, system performance, and the wider implementation context, this paper focuses specifically on the surgeon-level competencies required to interact safely with intraoperative AI guidance. Competency, AI literacy, human factors, and governance overlap, but this paper uses them in a specific sense. IAIL operates at the level of surgeon competency, meaning the ability to act safely on the basis of an understanding of how a system produces its output. It assumes AI literacy, draws on evidence from human factors, and is intended to function within governance structures rather than to replace them^[28].

These competencies apply to two distinct groups. Surgical trainees may increasingly encounter AI guidance during their formative years and could acquire these competencies progressively, alongside the operative skills they already develop through structured training programmes. Surgeons already in independent practice are in a different position. They may encounter these systems without having trained alongside them and, in many cases, without structured institutional preparation. The competencies proposed here are intended to apply to both groups, though the route to acquiring them may differ. For trainees, this may mean progressive integration into existing curricula through careful consideration of IAIL or other competency frameworks, if these are subsequently validated and adopted. For surgeons in practice, it may require continuing education and structured courses when these systems are introduced into clinical practice.

The relevance of these competencies may also extend across surgical specialities. A model trained to segment the total mesorectal excision plane can highlight areolar tissue as a visual landmark for the appropriate dissection plane^[19]. Whether that guidance is appropriate in an individual case remains a matter of surgical judgment informed by the wider operative and oncological context. A model trained to recognise anatomy during robot-assisted radical prostatectomy can identify relevant anatomical structures^[18]. Whether neurovascular bundle preservation is appropriate in an individual patient, however, depends on clinical factors beyond anatomical identification, including tumour stage, the estimated risk of extracapsular extension, and baseline erectile function^[29]. In each case, the model contributes anatomical information, while the surgeon contributes clinical judgment. The same distinction is apparent in these examples and may extend to other applications, although this has not been formally tested.

DISCUSSION

Most surgical AI research has focused on algorithmic performance, including accuracy, validation, and segmentation capability^[9,10]. Far less research has examined how surgeons respond to AI output: whether they trust it, question it, or set it aside^[10,30].

There is the question of what the surgeon can actually check. Deep learning models give an answer without showing how they arrived at it^[9]. A surgeon looking at an anatomical overlay produced by an AI model has no way to ask why and cannot question the answer in the way they would question an assistant. Whether surgeons will compensate by checking the anatomy themselves, or accept what they are shown because further checking takes effort, is not yet known.

The way model output is displayed may influence how it is received by the surgeon. A confident overlay, or a high confidence score displayed alongside it, may present the surgeon with an answer before they have formed their own judgment^[24,25]. In other high-risk fields, operators tend to place greater trust in an automated recommendation than in the same advice from a colleague, and experience does not appear to eliminate this tendency^[23,24]. Whether the same occurs in surgery has yet to be established.

Each IAIL domain responds to a specific practical challenge identified above. A case may involve a patient or presentation that was not represented in the data used to develop the model, so surgeons need to know the conditions under which a system was developed and validated. Inflammation or scarring may make the operative field look different from the training data, so surgeons need to judge whether the output fits the surgery in front of them. A model confidence score may look reassuring, but surgeons need to know what the score actually means. AI guidance may also conflict with the surgeon's own judgment, so surgeons need to be able to override it. Whatever decision is made, the surgeon must be able to account for it. Accordingly, the five domains in Table 1 were not simply drawn up first and then matched to problems. Rather, they emerged from the sequence of judgments a surgeon may need to make when deciding whether and how to use AI guidance in the operating room.

The final domain in Table 1, Clinical Accountability, deserves particular attention. If a model labels a structure with high confidence, the surgeon accepts it without independent verification, and the patient is injured, it may still not be clear where responsibility lies. Although governance and regulation remain under development in many jurisdictions, some current frameworks, such as the EU AI Act, assign oversight responsibilities to the hospital and the operating surgeon but say little about how responsibility should be divided when a surgeon follows plausible guidance that later proves to be wrong^[28,31]. In practice, much of that responsibility may still fall to the surgeon. Such responsibility is only fair if the surgeon was trained to judge the AI guidance in the first place. A surgeon who has never been taught to recognise when a model is operating outside the conditions under which it was developed and validated could therefore be held unfairly responsible for a judgment nobody prepared them to make. Training in these competencies is therefore not simply desirable: it is what allows the surgeon to carry that responsibility.

If the IAIL competencies are to be taught, they must also be assessed. The ability to override AI guidance is particularly difficult to evaluate in live practice because the situations that call for it are uncommon and cannot be scheduled. Simulation offers one approach. A simulated procedure in which AI guidance is deliberately incorrect at a defined point would allow the timing, accuracy, and reasoning of the surgeon's response to be recorded. Structured credentialing pathways, such as those established for robotic surgery, provide another possible approach, though their content would differ substantially^[21]. Neither approach has yet been tested for this purpose.

Embedding these competencies will not be straightforward. Curriculum time is finite, faculty would need to understand the material themselves, and validated tools for assessing these competencies have not yet been established. Institutions will vary in their ability to provide this training, and access to AI systems will differ between regions, so programmes will begin from different levels of readiness.

This proposal has several limitations. The clinical evidence on intraoperative AI remains concentrated in laparoscopic cholecystectomy, where computer vision applications have advanced furthest^[1-5]. Emerging work in colorectal and urological surgery suggests that comparable issues arise elsewhere, but the evidence remains preliminary and is derived from small, single-centre studies^[18,19]. Whether IAIL is transferable across other specialities, AI modalities, or operative contexts has not been tested. Furthermore, parts of the rationale for IAIL have been extrapolated from aviation and other high-reliability industries rather than derived directly from surgical data^[23-27]. The framework has also not undergone expert consensus, formal validity assessment, or clinical evaluation, and the questions it raises remain open. Addressing them will require prospective evaluation of IAIL across different operative environments, patient populations, and training systems.

CONCLUSION

IAIL is not a framework for doubt. It is a framework for responsible use.

This paper has proposed IAIL as a candidate competency framework for surgeons who may work with intraoperative AI guidance systems as they enter clinical practice. It is offered for testing rather than adoption. Whether the five domains proposed here are the right ones is a question for the field to answer. What is clearer is that the question needs to be asked now, while these systems are still being developed.