In a qualitative leap, researchers at Johns Hopkins University have taught the da Vinci surgical robot, powered by a ChatGPT-like model, how to successfully perform gallbladder removal surgery on pig organs. However, an unexpected obstacle stands in its way: corporate data secrecy.
Article contents:
- Introduction: when ChatGPT meets the surgeon’s scalpel
- From STAR to SRT-H: the evolution of surgical robots
- Imitation learning: how did the robot become a surgeon?
- Impressive results: human-matching precision
- Robotic secrets: the corporate data hurdle
- The workaround and future: from sensors to humanoid robots
- Conclusion: immense potential waiting for data to unleash it
Introduction: when ChatGPT meets the surgeon’s scalpel
In the late 1990s, Intuitive Surgical revolutionized the medical world by introducing da Vinci surgical robots, which allowed expert surgeons to perform remote operations with unprecedented precision. Now, Johns Hopkins University researchers are taking this technology to a completely new level, putting a ChatGPT-like artificial intelligence in the driver’s seat and teaching it how to perform a complex surgery on its own.
From STAR to SRT-H: the evolution of surgical robots
The idea of putting a computer behind the wheel of a surgical robot is not entirely new, but it has mostly relied on pre-programmed procedures. In 2022, a team led by Axel Krieger, an assistant professor of mechanical engineering at Johns Hopkins, built STAR (Smart Tissue Autonomous Robot), which successfully performed surgery on a live pig. But even STAR needed specially labeled tissue and a pre-defined plan.
The new robot, developed by Ji Woong Kim, a robotics researcher who led the study, is much more flexible. The new system, named SRT-H (Surgical Robot Transformer), relies on a standard da Vinci robot and software based on two “Transformer” models, the same architecture that powers ChatGPT. One model handles task planning while the other executes it precisely.
Imitation learning: how did the robot become a surgeon?
Kim and his team chose gallbladder removal for the robot to learn. They broke the procedure down into 17 steps, and obtained pig gallbladder and liver samples from pig cadavers to experiment on. Next, a trained research assistant operated the da Vinci robot, performing the procedure repeatedly to build a training dataset for the robot.
The algorithms powering SRT-H were trained on more than 17 hours of video captured from the da Vinci endoscope and cameras mounted on its robotic arms. This video was supplemented by kinematics data—the precise movements made by the robotic arms—and natural language feedback.
Impressive results: human-matching precision
Based on this data, Kim’s robot learned to perform the gallbladder removal procedure with a 100% success rate when working on samples it had not been trained on. It could also accept natural language human feedback—simple tips like “move your arm slightly to the left” or “place the clip a bit higher.” Compared to an expert human surgeon performing the same procedure, the robot was equally precise, albeit slightly slower.
Robotic secrets: the corporate data hurdle
However, the robot was not immune to corporate affairs. To move from working on pig cadaver samples to live pigs, and then to humans, robots like SRT-H need training data that is extremely difficult to obtain. Intuitive Surgical reportedly agrees to release video data from da Vinci robots, but the company does not release kinematics data. This is the exact data Kim says is essential for training the algorithms.
Kim said: “I know people at Intuitive Surgical’s headquarters, and I was talking to them. I was begging them to give us the data. They didn’t agree.” The explanation provided by Intuitive Surgical leadership is that they are worried competitors will reverse-engineer their robot mechanics. Kim argued: “It’s the upper management that is not keeping up with AI. They don’t realize the potential of these things.”
The workaround and future: from sensors to humanoid robots
Yet Kim already sees a way to overcome this problem. He said: “We can start attaching motion-tracking sensors to manual surgical tools, and obtain kinematics data that way.” Afterward, the movements of these tools guided by expert human surgeon hands can be replicated by traditional robotic arms.
And after that, Kim believes, we can go even further, toward science fiction. He said: “I’m currently at Stanford, and I’m very involved in a humanoid robotics project—building a general-purpose model. And one of the possible applications is in the operating room.”
Conclusion: immense potential waiting for data to unleash it
This research demonstrates the immense potential of artificial intelligence to revolutionize surgery, making it more precise, safe, and accessible to everyone. Nevertheless, it also highlights the major challenge of accessing the data needed to train these systems. The future of autonomous surgery may depend not only on the ingenuity of engineers, but also on the willingness of companies to share their data for the greater good.
Frequently asked questions
Will the robot replace human surgeons?
That is unlikely to happen in the near future. The current goal is to create a system that can assist surgeons and perform repetitive tasks with precision, allowing the human surgeon to focus on the most important and complex aspects of the operation.
What is “kinematics” data and why is it important?
Kinematics data is a record of the precise movements made by the robot’s arms. It is essential for teaching AI “how” to perform a task, not just “what” it is doing. Without it, it is like watching a video of a race car without knowing how the driver is moving the steering wheel and pedals.
How safe is this technology?
The technology is still in a highly experimental stage and requires many years of testing and verification before use on humans can be considered. Safety is the top priority, and multiple systems are being developed to ensure nothing goes wrong.
