AI-Based System for Assistance in Minimally Invasive Renal Procedures Using Mixed Reality. First Steps

Abstract

The main objective of this study is the implementation and configuration of an assistance system for minimally invasive renal surgeries, which incorporates an Artificial Intelligence (AI)-based automatic renal anatomy segmentation module using computed tomography (CT) imaging studies, as well as its integration into an immersive and interactive mixed reality (MR) system. The purpose is to enrich surgical planning, ensuring greater precision and safety to improve patient outcomes. The image dataset used in this study comes from the KITS23 challenge, from which 20 CT studies were randomly selected from the 489 available with ground truth annotations. The interactive MR interface was developed using Unity in combination with the Microsoft HoloLens v2 device. For medical image segmentation, the Vista3D AI model was employed due to its versatility and high performance. All studies were successfully segmented, showing a Dice score distribution with a high concentration of values above 0.8 for renal anatomy segmentation, indicating robust and consistent performance. However, in the case of cyst segmentation, the Dice score distribution revealed a significant proportion of low values, reflecting the complexity of this type of anatomical structure. Additionally, an application was developed for MR visualization of 3D renal anatomical models to facilitate surgical planning. This application allows clinicians to more accurately identify renal anatomy, complementing and improving traditional planning methods. The development of this assistance system lays the foundation for greater precision, error reduction and better surgical outcomes, contributing to safer and more efficient procedures.

Roberto Rodriguez-Echeverria

INTIA Director and Associate Professor

Associate Professor at the University of Extremadura. Software passionate, Deep learner, MTB rider and father of 2.