Changelog

Member Performance Now Supports up to 1000 Members

Analytics > Member Performance now supports selecting up to 1000 members per view, up from 50. The member list also follows your workspace or project selection, so you're choosing from the right people instead of your entire org.

Updated SDK with new KPIs

Analytics KPI data is now available through the API and the label-studio-sdk.

• The public API schema also now documents several new request fields.
• Some APIs have been deprecated (tagged "Deprecated"), and will not receive further updates or maintenance. Instructions to use a suitable alternative is provided for each deprecated endpoint.

Data Manager User Filters Now Scoped to Your Projects

Annotator, reviewer, and other user filters in the Data Manager now show people with access to your selected project instead of your entire organization. A filter also keeps showing people relevant to its specific column, including past contributors who later lost project access, so historical filters still resolve correctly.

Set Hotkeys Per Project

Override your keyboard shortcuts for an individual project without changing your account-wide defaults. Edit overrides for any project you have access to from My Account > Hotkeys.

Fixes and Improvements

• The Interfaces Web Agent now fails over across model providers automatically, so a rate limit or outage on one provider doesn't stop your agent from responding.
• Improved Data Manager filter and locked-tab experiences, including a read-only view of tab filters on locked tabs and preventing hidden columns from reappearing after unlocking a tab.
• Fixed vector region drawing being blocked when video vector labels exist on the same video
• Improved keyboard accessibility of Badge close buttons, including tab order and touch target size.

Filter Annotations by Annotators, Review Status, Comments, and more in Data Manager

Data Manager "Annotations" filters now support additional child filters. Filter by annotator(s), ground truth status, review outcome, reviewer, comment author, and select multiple values per filter at once.

• Combine several annotators, reviewers, or comment authors in one filter instead of switching between single-value filters.
• Works alongside the existing per-row AND/OR combination logic for annotation and prediction filters.

Right-Click Context Menus on Projects and Members Tables

Right-click any row in the Data Manager, the Organization Members table, or a project's Members page to get the same actions previously available only from the overflow menu, plus your browser's standard right-click options like opening a link in a new tab.

Fixes and Improvements

• Fixed the Data Manager multi-select user filter losing its search results after you picked a user.
• Fixed vector regions not displaying on video tasks after using Track Both, Stop, and Submit, and stopped LiDAR relation connectors from going stale after panning or zooming the bird's-eye view.
• Sorted Label Distribution projects newest first and corrected prediction and annotation total percentages so they reflect the right totals.
• Restored rejected tasks for paused annotators with Data Manager access when they're eligible for update.

Filter by Multiple Annotators and Reviewers at Once in Data Manager

Data Manager filters for Annotators, Reviewers, Comment Authors, Updated By, and Skipped By now accept more than one value at a time. Build a single view across several teammates instead of switching filters back and forth or exporting multiple views to compare them.

Filter a Wider Range of Annotation and Prediction Results

Data Manager annotation and prediction result filters now cover more granular dimension-level data, including results produced by Interfaces with per-row AND/OR logic for combining conditions. Narrow down to exactly the results you need to review or export.

Fixes and Improvements

• Fixed CSV import so columns are preserved when task data keys collide with control tag names.

Add or Update Columns in Task Data

As your project progresses, you may find that you need to add metadata columns or update fields along the way. Now, you can more easily add new data columns update values through the UI.

Additional fields (e.g. batch, priority, source, slice_id) can be used as filterable/sortable columns to more easily create the most useful task views.

Vertical Annotations Sidebar

Instead of having to scroll through different annotators on top tabs, you can now display annotations vertically on the left-hand side.

This is helpful when one task may have multiple annotations, or when you want to search for a specific annotator or label.

In a task, open up the Settings button on the bottom, and choose "Display annotations in vertical panel".

Bulk Review Action

Select multiple tasks and bulk Accept or Reject annotations from All Annotators or a certain Annotator.

This enables you to process reviews faster, especially from trusted annotators or high-agreement tasks.

Additional enhancements

• Filter for a much wider range of annotation and prediction results on Data Manager, including dimensions from Interfaces
• Export to CSV the list of active project members from the Project Settings > Members page
• Manager roles and above can Lock a Data Manager tab so its filters and columns stay fixed
• Admins can let people from a verified email domain join the organization through your existing SSO, without a separate invite for each person

Label Studio SDK 2.1.0

SDK 2.1.0 is now available on PyPI. This release focuses on stronger typing across the SDK, along with expanded response objects and new filtering options.

• Roughly 100 typing fixes in the public API. Fields that were previously typed as strings, such as pause, members, model_params, and workspace, now use proper object, boolean, integer, and number types in requests and responses.
• Enum types now have readable names (for example, Role9e7Enum is now OrganizationRoleEnum), and all id path parameters are properly typed as integers.
• New optional query parameters for filtering across several endpoints, including ML backends, organization memberships, predictions, projects, annotation history, and compatible projects for Prompts.
• Richer response schemas for several endpoints, including project updates, token rotation, and FSM backfill.

For full details, see the release notes on GitHub.

Video object segmentation and tracking for SAM 2

There is a new VideoVector tag that brings Segment Anything 2 (SAM 2) video segmentation and tracking into your annotation workflows.

With SAM 2 connected, you can segment an object once using points or bounding boxes, and the model will track it across frames as the video plays, even when the object moves in and out of frame. This removes the need to segment objects frame by frame.

To get started, you will need an ML backend (an open-source example is available) and GPU-backed hosting for your model. Then, in Label Studio:

• Build your labeling config with the VideoVector tag
• Connect your model
• Enable pre-annotations and auto-annotation

For more information, see the blog post or watch the 2-minute video tutorial.

Dashboards for projects

There are now two new dashboards available for projects: Throughput for tasks progress and Data Quality for agreement and label distribution. The Members dashboard has been updated with additional dimension-level metrics and reviewer work, and it is now also available under the Dashboards tab at the project-level.

Learn more in the docs

Vibe Code Labeling and Evaluation Interfaces

We're introducing Interfaces, an agentic builder for creating highly customized, multi-modal labeling UIs that you can reuse across projects.

Instead of selecting a template and tuning XML tags, use natural language to describe the labeling experience, and Label Studio generates a fully interactive interface that you can preview, refine, save, and share. With Interfaces, you can:

• Annotate data types that classic XML tags don't support, such as 3D files, GeoTIFF, and LiDAR
• Build custom logic directly into the interface component, rather than maintaining it separately as a plugin
• Reuse interfaces across projects as first-class objects, with built-in version history and one-click rollback

Review workflows, Agreements, Analytics, exports, and Prompts LLM-as-a-judge and pre-labeling workflows are supported with Interfaces.

Interfaces are available to users in the Owner, Admin, and Manager roles, and can be enabled or disabled for your organization from Organization > Settings > Interfaces.

For more information, see Interfaces.

Service accounts

You can now create service accounts for your organization.

A service account is granted programmatic access to Label Studio via our API, but cannot login or interact with the Label Studio UI.

By default, each organization can create one service account. This service account does not count towards your license's seat count.

You can create and manage service accounts from Organization > Settings > Service Accounts.

For more information, see Service accounts.

Filter against "is any of" and "is none of" value lists

You can now filter by is any of and is none of.

These filters accept value lists and can be used with the following fields:

• Task ID
• Inner ID
• task.data

Dependency updates

Made several security enhancements by updating key dependencies.

Performance improvements

Made performance improvements around image loading.

Review sampling

There is a new section under Project > Settings > Review that you can use to configure how many tasks should be sampled and sent to reviewers.

• Basic Sampling - Set a fixed percentage of eligible tasks to route for review.
• Agreement-based Sampling - Route tasks to review based on their agreement scores. For example, you can configure your project to review all tasks that have an agreement score less than 80%.

For more information, see Review Sampling.

Additional options for publishing projects

You can now publish projects by clicking the project state badge or from Settings > General.

Bulk assign members to projects or workspaces from the organization Members table

You can now bulk assign members to projects or workspaces from the Organization > Members table.

Indicator icon for agreement columns

If the settings for the Agreement columns in the Data Manager have been modified, you will now see a dot indicator icon.

Custom colors for workspaces and projects

You can now apply a custom color to project and workspace cards:

Project Members and Dashboard pages navigation change

As we continue to improve dashboards and analytics, the location of the project-level Members dashboard has changed. You can now find it alongside the project dashboard.

The project dashboard is now labeled Throughput.

Improved organization settings navigation

The Organization page navigation has been updated to consolidate settings locations.

• Member Tags and Activity Logs have been moved to the left-hand menu.
• The Usage & License page now only shows license and seat information.
• There is a new Security page for setting related to security:
  • Role Permissions (visible only to users in the Owner role, and previously located under Settings)
  • Access Token (previously located under Settings)
  • SSO & SCIM (previously located under Settings)
  • Session Timeout (previously located under Usage & License)
  • Storage Proxy (previously located under Usage & License)
• The Settings page now has the following:
  • Features (previously located under Usage & License)
  • Email Notifications (previously located under Usage & License)
  • Model Providers (no change)

Per-control-tag agreement included in task exports

You will now see a per_control_tag_agreements field in task exports, which maps each active control tag name to its agreement percent (0–100) for the task.

Publish projects from the breadcrumbs menu

Publishing controls for projects have been moved to the breadcrumbs menu.

To publish a project, click the pause icon ▶️ in the breadcrumbs menu.

Cloud storage visibility during project setup

When creating new projects, the option to set up cloud storage is more prominently visible with a link to the cloud storage setup workflow.

Performance improvements

Improved performance for filtering and sorting per-control-tag agreement.

Improved performance when filtering for compatible projects when creating a Prompt.

PDF labeling enhancements

Per-page classification

If you are using a tag that supports the perItem="true" parameter (for example, Choices, Rating, Taxonomy, Number, TextArea), you can now apply those annotations to individual PDF pages.

Page thumbnails

You can now view thumbnails for each page. These thumbnails will also provide information about any page-level annotations.

Configure ReactCode permissions

Administrators can now configure ReactCode permissions from Organization > Usage & License.

From here you can configure whether to enable ReactCode or whether to restrict it so that it is only allowed in external app mode.

You can also configure allowed origins and which browser features that iframes can access.

Expanded and improved agreement functionality

We have made multiple changes and improvements to agreements.

Introducing Consensus methodology

You can now select between Pairwise Averaging and Consensus methodology when calculating agreement.

Previously, all agreement calculations used Pairwise methodology.

• Consensus methodology calculates the percentage of annotators who chose the most common answer.
• Pairwise Averaging methodology calculates the average agreement between each pair of annotators.

You can learn more about each type of methodology in our documentation: Methodology.

Configure agreement metrics for each control tag

Under Settings > Quality > Agreement Columns, you can now select which metric you want to use for each individual control tag.

Before:

Previously, if you had multiple control tags in your labeling interface, you were limited to either the Exact Match metric or the Basic Matching metric, which would apply the default metric to each control tag.

After:

Now you can select a built-in metric for each control tag.

Configure which classes should be included in agreement calculation by control tag

You can also now decide which classes within each tag should be included when calculating agreement.

Under Settings > Quality > Agreement Columns, control tags that have classes (e.g. Choices and Labels) lists each class next to a checkbox.

You can deselect the checkbox to remove that class from the agreement calculation for that control tag. This also removes it from the overall agreement calculation for the task.

Set custom weights only at the tag level

As mentioned in an earlier changelog, how weights are configured for agreement has changed.

When you customize the weight for a control tag, you are deciding how much it should factor into the overall task agreement calculation. This lets you de-emphasize tags that should carry less weight when deciding task agreement.

Before:

Previously, you could configure weight by control tag and classes within the control tag.

After:

Now, you can only configure weight by control tag. (However, as mentioned above, you can remove classes from agreement calculation at the tag level instead.)

View agreement for each control tag

The Data Manager now has columns for each control tag. This has replaced the Agreement (Selected) column.

You can use the agreement columns to select which annotators and/or models to use in the agreement calculation.

For more information, see Agreement columns.

Deprecated metrics

As mentioned in a previous changelog, the following agreement metrics for rectangles, brushes, and polygons have been removed:

• F1 score at specific IOU threshold
• Recall at specific IOU threshold
• Precision at specific IOU threshold

Any existing projects that use the agreement metrics listed above automatically updated to use standard IoU calculation.

Additional resources

Task agreement documentation

Manage Quality at Scale with Super-granular Agreement

Video - Understanding Agreement Metrics with Thresholds

Video - Understanding Agreement | Consensus vs. Pairwise

Tag organization members

You can now add metadata tags to organization members. This can be useful if you need to keep track of member location, language, or other attributes.

You will now see:

• A new Tags column in the organization Members table, which you can use to view and apply tags.
• A new Tag Members button on the Organization page, which you can use to import a CSV file to bulk create and apply tags.
• A new Organization > Settings > Member Tags page where you can manually create tags or delete tags.

For more information, see Tag members.

SSO/SAML and SCIM UX enhancements

The SSO/SCIM settings are now located under Organization > Settings > SSO & SCIM.

These settings have also been improved to use a more guided setup process with automatic XML/URL metadata field mapping.

Label Distribution dashboard

There is a new Analytics > Label Distribution dashboard that provides insight into how labels are distributed within a project.

For more information, see Label Distribution dashboard.

Save analytic dashboard views

When viewing the Overview and Member Performance dashboards, you can now save your filters as a view.

For more information, see Save views.

Quickly resize columns in the template builder

You can now double-click to quickly resize panels in the template builder.

New custom permission for the Members dashboard

A new custom permission is available to control who can access the Members dashboard for a project: Access Project Members Dashboard

Updated project list view

The project list view now includes the project ID and the project description has been moved below the project title.

Performance improvements for image loading

Added staggered requests to improve performance when loading a large number of images from source storage.

Data Manager filter enhancements

Data Manager filters have the following enhancements:

• New copy and paste actions that you can use to copy filters between tabs within the same project or to tabs within other projects (any inapplicable fields will be automatically removed when pasting between projects).
• The ability to copy filters as JSON so that you can edit them elsewhere.
• The three most recent filter selections will appear at the top of the drop-down menu.
• When adding new filters, it will pre-populate with your most recent selections.

Performance improvements for task switching

Improved performance when clicking between different tasks in the Data Manager to open the Quick View interface.

Upcoming changes to the Agreement (Selected) column

To support upcoming enhancements to agreement calculations, we will be removing the Agreement (Selected) column and replacing it by adding new functionality to the Agreement column.

While all the functionality offered in Agreement (Selected) will still be available going forward, any filters based on this column will be removed and will need to be recreated.

Project overview analytics

There is a new Analytics > Overview page that provides insight into metrics and progress across multiple projects and workspaces.

For more information, see Projects overview dashboard.

Annotate images with vector lines

There are two new tags for image annotation: Vector and VectorLabels.

You can use these tags for point-based vector annotation (polylines, polygons, skeletons).