Changelog

Feature updates

Prompts, our Enterprise tool for evaluating and refining LLM prompts, now supports multi-class text classification projects. This will enable you to label your text with more than one choice, allowing for more complex classification use cases.

This also means that projects with the choice="multiple" parameter set will now appear in the Target Project drop-down menu when creating a Prompt (assuming that the project meets all other eligibility criteria).

Bug fixes

• Fixed an issue where the instructions modal was not appearing for reviewers even though Show before reviewing was enabled in the project settings.
• Fixed an issue in which draft lead time could incorrectly inflate the lead time calculation for an annotation.
• Fixed an issue in which Data Manager drop-down menus were inaccessible in smaller viewports.
• Fixed an issue which caused the Label Config UI preview to display stale information.
• Fixed an issue in which the agreement score was not updating after a reviewer completed the Fix+Accept action.
• Fixed an issue where images were improperly spaced in the Annotator Agreement Matrix.
• Fixed an issue in which Label All Tasks would not respect filters that had been applied in the Data Manager.
• Fixed an issue in which users were not able to use Quick View to load tasks that included an external taxonomy.
• Fixed an issue that caused the time portion of the DateTimePicker in the project Dashboard to not display correctly.

New features

Video frame classification

You can now apply labels to video frames. Previously, we only supported per-video classification.

This new feature allows you to apply labels at a per-frame level. You can implement this feature using a new tag: <TimelineLabels>.

For more information, see New! Video Frame Classification.

Named Entity Recognition projects for Prompts

You can now select NER projects when creating a prompt. Previously, Prompts only supported Text Classification projects.

For more information, see Named entity recognition (NER) in our Prompts documentation.

Feature updates

• When creating regions that have start and end times (such as when annotating sections of an audio track), you will now see the duration of your selection under the Info tab.

• The Projects page header has updated to have a more compact design:
  
  - The search bar, projects drop-down menu, and create actions have all been consolidated on one line.
  
  - The Use Template action has been moved and is now available as a drop-down option within the Create Project button.
  
  - For better clarity, the Use Template action has been renamed Create from saved template.
  

• You will now see a progress bar when performing searches against the activity log to indicate that the search is still processing.
• For better clarity, the Sandbox workspace has been renamed Personal Sandbox.

Bug fixes

• Fixed a regression issue with BigInteger support in the Data Manager.
• Fixed an issue with the Activity Logs page where some options would become unavailable.

Feature updates

• Changed the default behavior of the Project Dashboard so that it no longer defaults to the “include time” option in the calendar.

Bug fixes

• Fixed an issue in which project-level roles could not be reverted once set.
• Fixed an issue where user limits were not being enforced when users were added via LDAP.
• Fixed an issue where setting a task agreement threshold incorrectly affected counts in the project Dashboard.

New features

JavaScript for labeling customization

You can now add JavaScript to your Label Studio projects to further customize your labeling experience.

Note that due to security precautions, custom scripts must be enabled for your organization before will see the Scripts option when configuring the labeling interface. Contact your account representative to request this feature.

For more information, see the following resources:

• Custom scripts documentation
• HumanSignal Blog - Transform Your Labeling Workflow with Custom Scripts
• Video demo - Announcing Custom Scripts for Label Studio

Feature updates

You can now use Azure OpenAI when creating Prompts. For more information, see Model provider API keys.

Bug fixes

• Fixed an issue where users were seeing a runtime error when loading the dashboard in situations in which the project’s labeling configuration did not include labels.

Feature updates

• There is a new Performance Score column on the project Members page.
  
  The Performance Score column reflects the overall performance of annotators. This score takes into account all review actions, including annotations that were initially rejected and later accepted. The calculation is as follows:
  
  - Each annotation review action (Accept, Reject, Fix+Accept) contributes to the score.
  
  - The score is calculated by summing the scores of all review actions and dividing by the total number of review actions. For example:
  
  - If an annotation is rejected twice and then accepted once, the Performance Score would be (0 + 0 + 1) / 3 = 33%.
  
  - If an annotation is rejected once and then fixed+accepted with a score of 42%, the Performance Score would be (0 + 0.42) / 2 = 21%.
  
  This is different from the Review Score, which only reflects the current accepted/rejected state of annotations.

• When you make changes to the labeling configuration and attempt to navigate away before leaving, you will now see a warning message prompting you to save your changes.

• You can now reorder tabs in the Data Manager by dragging and dropping them.

Bug fixes

• Fixed an issue with breaking Text/HyperText content that contains emoji when regions are added.
• Fixed an issue where the project summary was not being included when duplicating a project.
• Fixed an issue where tracing was breaking presigned URL requests.
• Fixed an issue where users were seeing an error when visiting the Annotator Performance Report page without first selecting a user. Instead, users should be redirected away from the page until a user is selected.
• Fixed an issue where workspaces could be deleted through the API even if they still contained projects.
• Fixed an issue where, when attempting to select multiple bounding boxes by pressing Command (or Ctrl), a new bounding box would be created instead.
• Fixed an issue with region labels were not displaying the index number if there was not label text.

Feature updates

• When viewing regions with the Show region labels option enabled, the region label will now include the same index identifier that you see in the regions list.
  
  Before:

After:

• When using the Annotator Performance Reports, you can now select whether you want to aggregate data by creation date or updated date.

Bug fixes

• Fixed an issue with agreement scores for Annotators being inaccurately calculated on the Members page within projects. This issue would appear when a Reviewer would reject an annotation and then later accept it.
• Fixed an issue where images were improperly resizing after loading due to how the original dimensions were set.
• Fixed usability issues seen in the Members table once the member user list grows large enough.
• Fixed an issue with displaying large integer numbers in the Data Manager.
• Fixed an issue with tooltips that was causing errors in the transition effects.
• Fixed an issue where Annotators where able to resolve comments made by Reviewers, when this action should not be available to them.
• Fixed an issue where annotation history was not working correctly if created_by was null.
• Fixed an issue affecting SAML users caused by changes to the HumanSignal app domain.
• Fixed an issue where the docs link icon was not properly formatted.

Bug fixes

• Fixed an issue where users were unable to export in Pascal VOC XML format when applying bounding boxes to images.
• Fixed an issue where usernames in the Annotator Performance Report were not displayed correctly if the user had a long email address.
• Fixed an issue where annotation history was not working correctly if created_by was null.
• Fixed a sizing issue affecting the icons for workspace actions.
• Fixed an issue where after an annotation had been fixed and accepted, the update action was not recorded when exporting the annotation history to JSON.
• Fixed an issue in which sometimes the Submit button would be displayed when it should be the Update button.

New features

Introducing the Prompts tool

The Prompts tool leverages ChatGPT to help you evaluate and refine your LLM prompts. You can also use Prompts to generate predictions for automating your labeling process, and to quickly bootstrap labeling projects.

For more information, see Automate Data Labeling with HumanSignal and our Prompts documentation.

Refreshed Label Studio interface

The Label Studio UI has been upgraded with updated colors and fonts, giving it a sleek new look while maintaining the same intuitive navigation you're familiar with. All Label Studio tools, features, and settings are still in the same place, ensuring a smooth transition.

Feature updates

Improved performance on the Projects list page due to improvement on the API level.

Breaking changes

Fixed an issue with Google Cloud Storage when the connection has the Use pre-signed URLs option disabled. In these situations, Google was sending pre-signed URLs with the format https://storage.googleapis.com rather than sending BLOBs.

With this fix, Google Cloud Storage will begin returning BLOBs/base64 encoded data when Use pre-signed URLs is off. This means that Label Studio will start reading data from Google Cloud Storage buckets, which can result in large amounts of data being sent to your Label Studio instance - potentially affecting performance.

Bug fixes

• Fixed an issue where Redis storage connections were causing errors due to a missing field in the storage form (Storage Title).
• Fixed an issue where annotators were not being prompted to leave a comment when skipping a task, even though the project settings required them to do so.
• Fixed an issue where connected ML backends were unable to return more than one prediction per task.

Feature updates

When using the annotator performance report, you can now filter by project or workspace.

Bug fixes

• Fixed an issue where sometimes actual usernames were being replaced by a generic “Admin” username in the annotation history.
• Fixed an issue which caused the annotator dashboard project selector to not initialize on first interaction.
• Fix Redis connection with SSL.

Feature updates

• When you create a new snapshot, you will now have the option to include annotation history:

• For organizations with the HIDE_STORAGE_SETTINGS_FOR_MANAGER environment variable set to True, Managers will now be able to sync data from external storage as necessary rather than request assistance from an Admin user.

Bug fixes

• Fixed an issue where regions were disappearing when a user would switch between annotations and predictions.
• Fixed an issue where the Batch Predictions action was failing under certain conditions.
• Fixed an issue where tooltips on the dashboard were causing confusing mouseover behavior even when not visible.
• Fixed an issue with button sizing inconsistency when selecting date ranges.
• Fixed an issue that prevented users from fetching the URL for TimeSeries objects.

Feature updates

There is a new setting that can restrict users from uploading data directly to Label Studio, forcing them to only use cloud storage. If you would like to enable this setting, set the DISABLE_PROJECT_IMPORTS environment variable to True.

Bug fixes

• Fixed an issue with environment variables that have the prefix LABEL_STUDIO_ appearing in context logs.
• Fixed an issue where the progress bar for a project was not properly accounting for tasks that required additional annotators due to low agreement settings.
• Fixed an issue where multiple errors would appear in the console when users navigated away from the Data Manager.
• Fixed an issue where SCIM was not handling adding or disabling users in situations where the user limit was exceeded.
• Fixed an issue where errors were appearing in the user console related to loading large JS files.
• Fixed an issue where SCIM was creating duplicate users in situations in which there were discrepancies in casing used in their domain names.
• Fixed an issue where, if there was no name provided for a cloud storage connection, the project settings page would not load.
• Fixed an issue in which users who have their own organization could not be added to another organization using SCIM.
• Fixed an issue where logs were failing because the default level was set to DEBUG. The default log level is now INFO.