An international study led by University College London has revealed that neurosurgeons prefer statistical certainty scores and maps when using artificial intelligence during surgeries.
Visualization challenges in endoscopic pituitary surgeries and the importance of explanation
Comparison of five approaches for explaining surgical AI outputs
Survey methodology and the problem of confidence calibration in algorithmic models
The difference between interpretability and actual accuracy, and upcoming research steps
Frequently asked questions
Visualization challenges in endoscopic pituitary surgeries and the importance of explanation
An international research team led by University College London conducted a scientific survey involving 47 neurosurgeons to explore how artificial intelligence-based decision support systems should explain their identification of delicate anatomical structures during surgical procedures. The peer-reviewed study, published in Scientific Reports on October 24, 2026, focuses on endoscopic pituitary surgeries as a high-risk surgical example. It is worth noting that the study evaluates human physicians’ preferences regarding explanation and interpretation methods, and does not report clinical outcomes for patients from actual operations inside operating rooms.
In endoscopic surgical procedures, the surgeon needs to accurately distinguish between tissues and anatomical structures under complex conditions that may include poor visibility, unusual anatomical variation among patients, and a narrow visual field. Although a software algorithm can produce a color overlay that outlines a specific anatomical structure on the screen, this color highlighting alone does not show the surgeon how reliable that algorithmic prediction is.
Comparison of five approaches for explaining surgical AI outputs
The researchers compared five different approaches for presenting algorithmic explanations: certainty scores, certainty maps, visual saliency maps, scene similarity scores, and nearest neighbor illustrations. A certainty score summarizes the model’s numerical confidence level, while a certainty map displays spatial variations in the estimated confidence level across parts of the image. In contrast, visual saliency methods attempt to show the visual regions that influenced the model’s decision, whereas similarity techniques compare the current surgical scene with other reference images or representations.
The results showed that participating surgeons generally preferred certainty-based explanations, considering them more useful and practical when making real-time surgical decisions. Conversely, visual saliency and similarity methods were viewed as less directly related to immediate surgical navigation, although they might hold greater utility in medical training or academic education. These findings reflect surgeons’ responses to presented examples and preference metrics, rather than practical proof that a specific explanation method makes surgical procedures safer.
Survey methodology and the problem of confidence calibration in algorithmic models
The research paper utilized ranking systems, a modified explanation satisfaction scale, and open text feedback. The participant pool included surgeons working at medical institutions distributed across multiple countries. Because the cross-sectional survey relies on a small, self-selected sample, it cannot prove how all neurosurgeons will behave when using the same tool, nor does it demonstrate that displaying certainty indicators changes surgeon behavior in a clinically desired direction.
A critically important technical caveat also emerges here: a model’s confidence score is not necessarily accurately calibrated with reality, as an artificial intelligence system can express a high degree of certainty while being wrong at the same time, particularly when encountering unfamiliar anatomy that differs from its training data. Therefore, displaying numerical certainty is only useful if the estimates actually correspond to empirical error rates and physicians are properly trained to interpret them.
The difference between interpretability and actual accuracy, and upcoming research steps
Based on these data, the authors call for future evaluations of confidence calibration and clinical performance across various surgical specialties. This study contributes to the fields of human-computer interaction and medical AI design by identifying what experienced users say they need before relying on an algorithmic assistant.
For health and medical journalism, the fundamental distinction lies between interpretability, ease of use, actual model accuracy, and improved patient clinical outcomes; none of these elements automatically leads to the realization of the others. A popular and favored explanation feature may be associated with overconfidence in the physician, while a less intuitive feature might sometimes help detect errors. This makes the next step conducting a controlled study comparing decisions made with and without each explanation format, including realistic failure cases and appropriate human intervention.
Frequently asked questions
Question: What are the five methods compared by the study for explaining surgical AI?
Answer: The study compared certainty scores, certainty maps, visual saliency maps, scene similarity scores, and nearest neighbor illustrations.
Question: Which type of explanations did neurosurgeons prefer during surgeries?
Answer: Surgeons preferred certainty score and map-based explanations for immediate surgical navigation, while viewing saliency and similarity maps as more suitable for training and education.
Question: What are the technical risks associated with relying on a model’s certainty score?
Answer: A model’s confidence score may not be accurately calibrated, as the system can show high certainty despite being wrong, especially when encountering anatomical structures that differ from training data.
Question: Did the study prove an actual improvement in patient safety inside operating rooms?
Answer: No, the study relied on a survey of 47 surgeons’ preferences regarding presented examples, and did not test actual patient outcomes in live surgical operations.
