Like many parts of the ML/AI pipeline, the use cases and approaches to Data Labeling have been rapidly evolving with the introduction of LLMs. In the last few months, a significant development in the dynamic world of AI caught our attention — the rise of agents (such as AutoGPT has shown us). We've delved deep into this topic on our blog and, simultaneously, embarked on an exploration to grasp the mechanics and potential of these agents applied to various data processing tasks, starting with the problem of data labeling.
Agents combine knowledge outputs from LLMs and action on them in production systems, thus their reliability to correctly and consistently perform operations is mission critical. As with many in industry attempting to use agents in real-world applications, we had the need to ensure reliability—beyond monitoring and observability—with the ability to guide agents with human feedback. We saw an opportunity to create a new agent framework that could dramatically increase the efficiency of data labeling (and broader application across data processing tasks), with the unique ability to be guided by human signal.
After significant effort to arrive at a successful prototype, we are excited to introduce Adala, the Autonomous DAta Labeling Agent. Upholding our open source principles, and encouraged by the large community that’s grown around Label Studio, we were committed to open sourcing the technology from the start.
Adala is a robust framework for implementing agents that specialize in advanced data processing tasks in data labeling and generation. Think of these agents as smart assistants with the novel ability to teach themselves and develop specific Skills, such as data classification, summarization, or data generation.
The agent’s learning is shaped by the context you set for it and the data they encounter. For Adala, you set this context by giving them a ground truth dataset—Label Studio is of course one option to build that ground truth dataset. As the environment evolves—perhaps with additional data or insights—agents further refine their skills. Skills are fundamentally versatile, adaptable, and composable, enabling agents to handle intricate tasks with ease.
The basis for this functionality derives from the agent Runtime – an LLM where the agent's code executes. We will support OpenAI to start, but we’ll be actively working to support more runtimes (ideally through contributions and collaborations).
To ensure our agents remember and build upon their experiences, Adala provides a Memory component—a dynamic storage space for the agent's acquired knowledge. For instance, retrieving the previous experiences of an agent’s errors (and subsequent human feedback) allows them a starting point from which to branch off into learning or improving skills.
Explore AdalaTo allow Adala to produce reliable agents, we devised two main strategies:
We believe that true progress in the field of AI comes when knowledge is accessible and collaborative, and there’s a strong feedback loop. By releasing Adala with an OSI-approved open source license, we hope to inspire creativity and new applications we couldn’t have imagined, as well as drive important standards and best practices in a rapidly changing market.
We've designed Adala with modularity at its core, emphasizing our belief in strong contributions from the community. We eagerly invite the AI community to contribute by:
We believe Adala has the potential to reshape the landscape of data processing, training, fine-tuning AI models, and building AI applications. While the technology is early, we’re committed to helping our customers navigate the rapid pace of AI innovation and drive more efficiency into their data labeling processes.
We're hosting a livestream to show you how Adala applies Generative AI Agents to “act, observe, and adapt,” learning from human feedback while independently completing labeling tasks. Join us to learn more:
Tuesday, November 7, 2023
11AM Pacific / 2PM Eastern
Large Language Models (LLMs) are revolutionizing how we tackle data annotation, offering speed and efficiency that were previously unattainable. With Label Studio's Prompts, you can now handle multiple natural language processing (NLP) annotation types for a single data task in a unified Label Studio workflow.
Annotator Performance Dashboards make it easer to manage labeling teams at scale and orchestrate all the internal and external resources needed to improve data quality, without sacrificing speed.
In this post, we’ll guide you through the process of using Prompts in Label Studio Enterprise to pre-annotate data for Named Entity Recognition (NER) tasks.