Video Object Segmentation and Tracking with VideoVector tag for SAM 2 in Label Studio

Segment Anything 2 (SAM 2) lets you segment any object within a video and follow that object as it moves, eliminating the need to segment objects frame by frame.

With the new VideoVector tag in Label Studio, you can take full advantage of SAM 2’s video segmentation tracking capabilities in your annotation workflows.

It’s easy to get started. All you need is an ML backend (we have an open-source example you can use) and somewhere to host your model with GPU availability.

Then, in Label Studio, you’ll:

1. Build your labeling config with the VideoVector tag
2. Connect your model with our without authentication, and
3. Enable pre-annotations and auto-annotation

Once configured, you can use Label Studio points or bounding boxes to segment objects. As you play the video, you’ll see that SAM 2 accurately recognizes the object in motion as you track forward and back, even if the object slips in and out of frame.

Watch the VideoVector tag tutorial

Here’s a short video that demonstrates the setup in Label Studio:

More ways to use SAM 2 with Label Studio

• Using SAM 2 for image annotation
• How to build a labeling tool for SAM2 interactive precision segmentation with prompt refinement in label studio

VideoVector unlocks the part of SAM 2 that’s most powerful for anyone doing video annotation work: continuous tracking. If your project involves following people, vehicles, or equipment through footage, this cuts out most of the manual labor that used to eat up your annotators' week.

VideoVector is available in Label Studio Starter Cloud and above today. Give it a try on your next video project!