Accelerate Labeling and Model Evaluation with Prompts—Now Generally Available
Sheree Zhang
We're excited to announce Prompts is now generally available (GA) in Label Studio SaaS!
With Prompts, data teams can take advantage of LLMs to accelerate labeling without sacrificing quality. Data scientists can remove bottlenecks and bootstrap projects for faster insights. Subject matter experts can spend their time reviewing outputs instead of labeling manually. Prompts makes all of this available with a friendly interface integrated in your existing Label Studio workflows.
If you're on a Starter Cloud or Enterprise SaaS plan, you can start using Prompts today—no setup required! If you’re not yet using a Label Studio SaaS plan, you can start a trial of Label Studio Enterprise to get access to Prompts today.
After months of refining and expanding its capabilities based on feedback from our initial users, Prompts now supports more use cases, more model integrations, and more features.
Prompts now seamlessly integrates with a variety of foundation models, so you can compare and choose the best LLM for your labeling tasks. Supported models and model providers include:
Prompts supports a variety labeling use cases, including:
Prompts now includes features that unlock comparison and evaluation use cases for data teams:
Prompts makes LLM-powered automation scalable for subject matter experts (SMEs) and machine learning teams within human-in-the-loop workflows.
Accelerate annotation efficiency by generating high-quality initial labels for human review. Rapidly pre-label data to bootstrap new projects or push predictions into a labeling project.
Compare outputs from different models and user prompts against ground truth to evaluate models or optimize prompt effectiveness. Determine which models deliver the highest quality results with lowest costs based on your unique project attributes.
Use LLMs to create new labeled examples to expand datasets, balance underrepresented classes, or form Q&A pairs for retrieval-augmented generation (RAG) systems
Prompts simplifies the process of building, testing, and iterating on LLM-powered labeling workflows by integrating an agentic AI approach. Powered by Adala, our open-source agentic AI framework, Prompts enables users to use natural language to define their task, select an LLM, generate predictions, and enhance (or auto-enhance!) performance based on real-time quality metrics. Fully embedded into the Label Studio interface, Prompts ensures a seamless experience from data ingestion to LLM predictions to human-in-the-loop review.
Early users of Prompts are already seeing transformative results in their annotation workflows:
✅ Faster insights across large datasets—reducing labeling time by 5-10x
✅ Near-human accuracy in auto-labeling tasks
✅ Empowered SMEs to leverage LLMs in their workflows without requiring ML expertise
For a limited time, Prompts is free to try on any Label Studio SaaS plan, Enterprise or Starter Cloud. Reach out to us or contact your customer success manager for any questions.
📅 Join a live workshop to see Prompts in action and get expert guidance:
📖 Read the docs and get started today:
With Label Studio's Prompts, you can now handle multiple image annotation types - from basic object classification to nuanced caption generation - in a unified workflow that maintains both efficiency and quality.
Large Language Models (LLMs) are revolutionizing how we tackle data annotation, offering speed and efficiency that were previously unattainable. With Label Studio's Prompts, you can now handle multiple natural language processing (NLP) annotation types for a single data task in a unified Label Studio workflow.
In this post, we’ll guide you through the process of using Prompts in Label Studio Enterprise to pre-annotate data for Named Entity Recognition (NER) tasks.