Porch Group rapidly prototypes new models faster

Introduction

Founded in 2012, Porch Group is a technology company with a hub-and-spoke model connecting homeowners, services, software, and insurance. Part of Porch Group’s mission is providing data-driven solutions to automate and optimize underwriting, contracts, inspections, warranties and product recalls.

At Porch Group, the machine learning (ML) team sits at the center of these connected business units. Their purpose is to identify opportunities, build custom data models, and ultimately provide compound value to consumers across Porch Group’s companies and product lines.

Compared to standalone insurance or home service providers, Porch Group’s ability to connect data across contexts is an advantage. They can use proprietary data models to realize insights that translate into useful features for customers.

For example, one of the custom models developed by the team reads serial numbers from photos of home appliances so they can monitor manufacturing recalls and notify homeowners if they’re affected.

However, before the team can validate a model is worth building, they usually need to label a large quantity of data upfront. They might have a hypothesis about a data project to benefit the insurance side of the business, but first the team needs to process text from millions of individual PDF reports.

With Label Studio Enterprise, the ML team can put their review process on rails and label large amounts of data to discover whether a product is worth pursuing.

Problem

Risky Spreadsheets and Broken Workflows

Before adopting Label Studio, Porch Group’s ML team struggled with manual, error-prone data workflows which slowed down their ability to develop and validate new models. Previously, the whole team was working together out of a large-sized Google Sheet. Accidental data loss was a regular concern. A single misclick could wipe out hours of work.

Kim Cardenas, Associate Project Coordinator, recalls, “It was hectic. I was scared that if anybody moved anything in the Google Sheets, the information may be deleted.”

Even once review processes were improved, identifying relevant data for model training took a lot of time. The team might budget an entire week to manually label and gather enough examples for a balanced, high-quality data set.

Solution

From a Week-long Process to a Single Day Workflow

After moving to Label Studio Enterprise and adopting the Prompts feature, Porch realized a dramatic improvement in their workflows.

• Faster Labeling and Time to Insights: The manual process that previously took one week for initial research can now be done in a single day, 5x faster. When working with Prompts, the team can label 10x more attributes in the same amount of time compared to manual annotations.
• Pre-labeling and Filtering for Improved Efficiency: The Porch team wants a balance of 1,000 ‘yes’ labels and 1,000 ‘no’ labels for high-quality datasets. Previously, finding low-frequency labels (e.g., appearing in 15-20% of tasks) was time-consuming. Now Porch can use AI to classify and filter data, identifying relevant examples much faster.
• Reduced Costs for AI Model Execution: If the Porch team had wanted to run LLMs on the full-text reports without any pre-filtering, they would have incurred significantly higher costs. Imagine processing millions of reports each containing 35+ pages. Instead, they can use a two-step filtering approach: first, regex narrows down potential snippets. Then, they use Prompts to refine those selections, reducing computational costs.
• Better Review Mechanisms for Quality Control: Label Studio’s built-in review workflows with distinct role types allows Porch to systematically validate results from AI-generated labels, always keeping humans in the loop. Now instead of spending time manually labeling, the team can switch their efforts to reviewing for accuracy. They can easily filter for rejected reviews or open comments and discuss until they reach a resolution.

Thomas Busath, a machine learning engineer on the Porch team said, “One of the biggest benefits of Label Studio Enterprise for us has been the review mechanism.”

All together, the team has been able to improve the accuracy of annotations and get better coverage while moving faster and maintaining a better experience for people working on the project team.

“With Prompts in the HumanSignal platform, we can explore more ideas, faster. Before pre-labeling with Prompts, our old process required a full week for research. Now we can research ten ideas in that same time.”

Patrick Leonard

Senior Product Manager

Democratizing Data Annotation Wins with Prompts

When asked what they would do if Prompts were no longer available, the Porch team’s response was strong: they’d panic. The feature has quickly become an essential part of their workflow, enabling them to:

• Conduct AI-assisted research in a fraction of the time.
• Increase their labeling efficiency by over 500%.
• Reduce costs by avoiding manual annotations for datasets that would not get used.

With Prompts, even users without programming experience can effectively adopt AI-driven workflows for data annotation and insights.

Conclusion

Faster, Leaner, and Easier to Scale

With Label Studio Enterprise and Prompts on the HumanSignal Platform, Porch Group’s data team has unlocked more cost-efficient AI development while improving quality. Shifting away from cumbersome processes has allowed the team to deliver more value, faster.

If your organization is struggling with similar data labeling challenges, there’s hope: you can bring Prompts into your labeling workflows to make them more scalable and accessible.

Following the advice of the Porch team, “Start with simple projects” and automate wherever possible.