[{"data":1,"prerenderedAt":4},["ShallowReactive",2],{"readme:sam2":3},"\u003Ch1>SAM 2: Segment Anything in Images and Videos\u003C\u002Fh1>\n\u003Cp>\u003Cstrong>\u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fresearch\u002F\" rel=\"nofollow ugc noopener\">AI at Meta, FAIR\u003C\u002Fa>\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cp>\u003Ca href=\"https:\u002F\u002Fnikhilaravi.com\u002F\" rel=\"nofollow ugc noopener\">Nikhila Ravi\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fgabeur.github.io\u002F\" rel=\"nofollow ugc noopener\">Valentin Gabeur\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=E8DVVYQAAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Yuan-Ting Hu\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fronghanghu.com\u002F\" rel=\"nofollow ugc noopener\">Ronghang Hu\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=4LWx24UAAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Chaitanya Ryali\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=VeTSl0wAAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Tengyu Ma\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fhkhedr.com\u002F\" rel=\"nofollow ugc noopener\">Haitham Khedr\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.de\u002Fcitations?user=Tpt57v0AAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Roman Rädle\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?hl=fr&user=n-SnMhoAAAAJ\" rel=\"nofollow ugc noopener\">Chloe Rolland\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.com\u002Fcitations?user=c8IpF9gAAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Laura Gustafson\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fericmintun.github.io\u002F\" rel=\"nofollow ugc noopener\">Eric Mintun\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fjunting.github.io\u002F\" rel=\"nofollow ugc noopener\">Junting Pan\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fscholar.google.co.in\u002Fcitations?user=m34oaWEAAAAJ&hl=en\" rel=\"nofollow ugc noopener\">Kalyan Vasudev Alwala\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fwww.nicolascarion.com\u002F\" rel=\"nofollow ugc noopener\">Nicolas Carion\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fchaoyuan.org\u002F\" rel=\"nofollow ugc noopener\">Chao-Yuan Wu\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fwww.rossgirshick.info\u002F\" rel=\"nofollow ugc noopener\">Ross Girshick\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Fpdollar.github.io\u002F\" rel=\"nofollow ugc noopener\">Piotr Dollár\u003C\u002Fa>, \u003Ca href=\"https:\u002F\u002Ffeichtenhofer.github.io\u002F\" rel=\"nofollow ugc noopener\">Christoph Feichtenhofer\u003C\u002Fa>\u003C\u002Fp>\n\u003Cp>[\u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fresearch\u002Fpublications\u002Fsam-2-segment-anything-in-images-and-videos\u002F\" rel=\"nofollow ugc noopener\">\u003Ccode>Paper\u003C\u002Fcode>\u003C\u002Fa>] [\u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fsam2\" rel=\"nofollow ugc noopener\">\u003Ccode>Project\u003C\u002Fcode>\u003C\u002Fa>] [\u003Ca href=\"https:\u002F\u002Fsam2.metademolab.com\u002F\" rel=\"nofollow ugc noopener\">\u003Ccode>Demo\u003C\u002Fcode>\u003C\u002Fa>] [\u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fdatasets\u002Fsegment-anything-video\" rel=\"nofollow ugc noopener\">\u003Ccode>Dataset\u003C\u002Fcode>\u003C\u002Fa>] [\u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fblog\u002Fsegment-anything-2\" rel=\"nofollow ugc noopener\">\u003Ccode>Blog\u003C\u002Fcode>\u003C\u002Fa>] [\u003Ca href=\"#citing-sam-2\" rel=\"nofollow ugc noopener\">\u003Ccode>BibTeX\u003C\u002Fcode>\u003C\u002Fa>]\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Ffacebookresearch\u002Fsam2\u002FHEAD\u002Fassets\u002Fmodel_diagram.png?raw=true\" alt=\"SAM 2 architecture\" \u002F>\u003C\u002Fp>\n\u003Cp>\u003Cstrong>Segment Anything Model 2 (SAM 2)\u003C\u002Fstrong> is a foundation model towards solving promptable visual segmentation in images and videos. We extend SAM to video by considering images as a video with a single frame. The model design is a simple transformer architecture with streaming memory for real-time video processing. We build a model-in-the-loop data engine, which improves model and data via user interaction, to collect \u003Ca href=\"https:\u002F\u002Fai.meta.com\u002Fdatasets\u002Fsegment-anything-video\" rel=\"nofollow ugc noopener\">\u003Cstrong>our SA-V dataset\u003C\u002Fstrong>\u003C\u002Fa>, the largest video segmentation dataset to date. SAM 2 trained on our data provides strong performance across a wide range of tasks and visual domains.\u003C\u002Fp>\n\u003Cp>\u003Cimg src=\"https:\u002F\u002Fraw.githubusercontent.com\u002Ffacebookresearch\u002Fsam2\u002FHEAD\u002Fassets\u002Fsa_v_dataset.jpg?raw=true\" alt=\"SA-V dataset\" \u002F>\u003C\u002Fp>\n\u003Ch2>Latest updates\u003C\u002Fh2>\n\u003Cp>\u003Cstrong>12\u002F11\u002F2024 -- full model compilation for a major VOS speedup and a new \u003Ccode>SAM2VideoPredictor\u003C\u002Fcode> to better handle multi-object tracking\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>We now support \u003Ccode>torch.compile\u003C\u002Fcode> of the entire SAM 2 model on videos, which can be turned on by setting \u003Ccode>vos_optimized=True\u003C\u002Fcode> in \u003Ccode>build_sam2_video_predictor\u003C\u002Fcode>, leading to a major speedup for VOS inference.\u003C\u002Fli>\n\u003Cli>We update the implementation of \u003Ccode>SAM2VideoPredictor\u003C\u002Fcode> to support independent per-object inference, allowing us to relax the assumption of prompting for multi-object tracking and adding new objects after tracking starts.\u003C\u002Fli>\n\u003Cli>See \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsam2\u002Fblob\u002FHEAD\u002FRELEASE_NOTES.md\" rel=\"nofollow ugc noopener\">\u003Ccode>RELEASE_NOTES.md\u003C\u002Fcode>\u003C\u002Fa> for full details.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Cp>\u003Cstrong>09\u002F30\u002F2024 -- SAM 2.1 Developer Suite (new checkpoints, training code, web demo) is released\u003C\u002Fstrong>\u003C\u002Fp>\n\u003Cul>\n\u003Cli>A new suite of improved model checkpoints (denoted as \u003Cstrong>SAM 2.1\u003C\u002Fstrong>) are released. See \u003Ca href=\"#model-description\" rel=\"nofollow ugc noopener\">Model Description\u003C\u002Fa> for details.\u003Cul>\n\u003Cli>To use the new SAM 2.1 checkpoints, you need the latest model code from this repo. If you have installed an earlier version of this repo, please first uninstall the previous version via \u003Ccode>pip uninstall SAM-2\u003C\u002Fcode>, pull the latest code from this repo (with \u003Ccode>git pull\u003C\u002Fcode>), and then reinstall the repo following \u003Ca href=\"#installation\" rel=\"nofollow ugc noopener\">Installation\u003C\u002Fa> below.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003C\u002Fli>\n\u003Cli>The training (and fine-tuning) code has been released. See \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsam2\u002Fblob\u002FHEAD\u002Ftraining\u002FREADME.md\" rel=\"nofollow ugc noopener\">\u003Ccode>training\u002FREADME.md\u003C\u002Fcode>\u003C\u002Fa> on how to get started.\u003C\u002Fli>\n\u003Cli>The frontend + backend code for the SAM 2 web demo has been released. See \u003Ca href=\"https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fsam2\u002Fblob\u002FHEAD\u002Fdemo\u002FREADME.md\" rel=\"nofollow ugc noopener\">\u003Ccode>demo\u002FREADME.md\u003C\u002Fcode>\u003C\u002Fa> for details.\u003C\u002Fli>\n\u003C\u002Ful>\n\u003Ch2>Installation\u003C\u002Fh2>\n\u003Cp>SAM 2 needs to be installed first before use. The code requires \u003Ccode>python>=3.10\u003C\u002Fcode>, as well as \u003Ccode>torch>=2.5.1\u003C\u002Fcode> and \u003Ccode>torchvision>=0.20.1\u003C\u002Fcode>. Please follow the instructions \u003Ca href=\"https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F\" rel=\"nofollow ugc noopener\">here\u003C\u002Fa> to install both PyTorch and TorchVision dep\u003C\u002Fp>\n",1784240407992]