🏥 Medical-SAM3

A Foundation Model for Universal Prompt-Driven Medical Image Segmentation

  

📰 News

• [2026-01-20]: 🚀 Pretrained weights for Medical-SAM3 are released!
• [2026-01-15]: 📄 Paper is available on arXiv.

⚡ Inference & Evaluation

We provide a comprehensive toolkit to run inference on diverse medical datasets (e.g., CHASE_DB1, Synapse, etc.).

The inference pipeline supports:

• 📊 Model Evaluation: Run Medical-SAM3 on supported datasets with a single command.
• ⚖️ Baseline Comparison: Compare performance against the vanilla SAM3 or other baselines.
• 🖼️ Visualization: Generate and save segmentation masks for qualitative analysis.

Repository Layout

• Kvasir-SEG/ is local example data for smoke tests and demos.
• data/spatialLIBD/151673/tissue_hires_image.png is the current default single-image inference target for spatial transcriptomics experiments.
• data/spatialLIBD/151673/151673_full_image.tif is kept as the full-resolution source image.
• inference/run_spatiallibd_prompts.py runs text-prompted inference on that TIFF and saves one mask per prompt.

Current spatialLIBD workflow

The current repo workflow is centered on a single spatialLIBD TIFF image rather than the bundled medical evaluation datasets.

Current prompts:

• dorsolateral prefrontal cortex Layer1
• dorsolateral prefrontal cortex Layer3
• dorsolateral prefrontal cortex Layer6
• dorsolateral prefrontal cortex Layer4
• dorsolateral prefrontal cortex Layer5
• dorsolateral prefrontal cortex Layer2
• dorsolateral prefrontal cortex White Matter

📅 Todo List

Feature	Status	Description
Demo	🚧 Doing	Online interactive demo.
Data Scaling	🚧 Doing	Significantly expand the training corpus and evaluate on broader and more diverse medical datasets.
Training Code	📅 Planned	Release full training scripts and data construction guidelines.
Medical-SAM3 Agent	📅 Planned	Integrate LLMs to enable agentic reasoning and interaction for segmentation tasks.

📢 We are actively updating this repository. If you are interested in any features above, feel free to open an issue!

📝 Citation

If you find Medical-SAM3 useful for your research or work, please consider citing our paper:

@article{jiang2026medicalsam3,
  title={Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation},
  author={Jiang, Chongcong and Ding, Tianxingjian and Song, Chuhan and Tu, Jiachen and Yan, Ziyang and Shao, Yihua and Wang, Zhenyi and Shang, Yuzhang and Han, Tianyu and Tian, Yu},
  journal={arXiv preprint arXiv:2601.10880},
  year={2026},
  url={https://arxiv.org/abs/2601.10880}
}