Monitor Models in Production with Label Studio

You’ve already evaluated your LLM or trained and validated your ML model for quality, but there are a range of things that can happen in production, from weird model behaviors to unexpected human inputs, not to mention the impacts of our ever changing world on a model’s context.

And while there are many tools on the market that can help monitor model drift, as a data scientist or researcher, there’s no comparable replacement for having a human understanding what’s really going on once a model is in production.

Join us for a technical webinar, where ML Evangelist Micaela Kaplan will demonstrate how to implement a recurring model evaluation workflow with Label Studio. Learn how to seamlessly integrate human-in-the-loop (HITL) monitoring into your production pipeline, ensuring your GenAI/ML models maintain peak performance over time. She’ll show you how to:

• Use open source scripts to automatically sample real production data from model logs at the cadence you set (eg weekly)
• Leverage the Label Studio SDK to automatically set up a project and view the outputs as tasks
• Configure notifications for when your model logs are ready for review in Label Studio
• Correct inaccuracies, report on model quality, and leverage production labeled data for model retraining when necessary

The workflow outlined in this webinar will give you and your team confidence that you are safeguarding the quality of your models while providing a path for model retraining and improvement. Don’t miss out, register now!