Potential and prospects of segment anything model: a survey

Recently, the Journal of Image and Graphics published online the research findings of Professor Zhang Junping from the School of Computer Science at Fudan University. The study highlighted the rapid development of artificial general intelligence (AGI) research, propelled by the advent of foundational models such as contrastive language-image pre-training (CLIP), chat generative pre-trained Transformer (ChatGPT), and generative pre-trained Transformer-4 (GPT-4). AGI aims to endow AI systems with robust capabilities, enabling autonomous learning, continuous evolution, and the ability to tackle various problems and tasks, thus finding wide-ranging applications across multiple fields. These foundational models, after being trained on large-scale datasets, have successfully addressed diverse downstream tasks.

Within this context, Meta's Segment Anything Model (SAM) achieved a significant breakthrough in 2023, demonstrating exceptional performance in the field of image segmentation, earning it the moniker of the "terminator" of image segmentation. One contributing factor to this breakthrough is the SAM data engine methodology, which, through a three-stage process, curated the Segment Anything 1 Billion (SA-1B) image segmentation dataset, comprising 11 million images and over 1 billion masks, ensuring high-quality and diverse masks. Following the open-sourcing of SAM, researchers proposed a series of improvements and applications for the model.

To comprehensively understand the development trajectory, advantages, and limitations of the Segment Anything Model, this paper reviews and summarizes the research progress on SAM. Initially, it provides a brief overview of the background and core framework of the model from multiple aspects, including foundational models, data engines, and datasets. Building on this foundation, the paper meticulously reviews current improvement methods for the Segment Anything Model, focusing on two key directions: enhancing inference speed and improving prediction accuracy. Furthermore, it delves into the extensive applications of the model in image processing tasks, video-related tasks, and other fields. This section details the model's exceptional performance across various tasks and data types, highlighting its versatility and developmental potential in multiple domains. Finally, the paper conducts an in-depth analysis and discussion on the future development directions and potential application prospects of the Segment Anything Model.

See the article:

Potential and prospects of segment anything model:a survey

https://doi.org/10.11834/jig.230792