No More Training: SAM's Zero-Shot Transfer Capabilities for Cost-Efficient Medical Image Segmentation

Abstract

Semantic segmentation of medical images has enormous potential for diagnosis and surgery. However, achieving accurate results involves designing and training complex Deep Learning (DL) models specific to this task, something only within reach of a few. Segment Anything Model (SAM) is a model developed by Meta capable of segmenting objects present in virtually any type of image. This work shows the robustness and exceptional performance of SAM in medical image segmentation, even in the absence of direct training on this type of images (computed tomography (CT) scans of the lung and chest X-rays, in particular). Furthermore, it achieves these impressive results with minimal user intervention. Although the dataset used to train SAM does not contain a single sample of both types of medical images, processing a popular dataset composed of 20 volumes with a total of 3520 slices using the ViTL version of the model yields an average Jaccard index of 91.45% and an average Dice score of 94.95%. The same version of the model achieves a Dice score of 93.19% and a Jaccard index of 87.45% when segmenting a frequently used chest X-ray dataset. The obtained values are above the 70% mark recommended in the literature and close to state-of-the-art models specifically developed for medical segmentation. These results are achieved without user interaction by providing the model with positive prompts based on the masks from the dataset used and a negative prompt located at the center of the bounding box containing the masks.

Roberto Rodriguez-Echeverria

INTIA Director and Associate Professor

Associate Professor at the University of Extremadura. Software passionate, Deep learner, MTB rider and father of 2.

Fernando Sánchez-Figueroa

Full Professor

My research focuses on web engineering, big data visualization, and MDD.