Charles Wu
← HomeLivePerson · AI tooling
04

86% faster training-data labeling, with 15% fewer errors

Spreadsheet labeling bottlenecked model training and introduced fatigue errors. I designed a keyboard-first annotation workflow built for long, repetitive labeling sessions.

1 weekTo full-team adoption
86%Faster classification vs. spreadsheets
15%Fewer annotation errors vs. spreadsheets
TeamProduct designer · team of 4
When2019
Problem
Spreadsheet labeling slowed model training and introduced fatigue errors into the data.
My contribution
Owned annotator research, interaction design, and validation with a four-person team.
Key decision
Optimized for keyboard control, low visual noise, and confirmation guardrails instead of adding more features.
Downstream effect
Model-training cycles moved from quarterly to monthly, and the full team adopted the workflow within one week.
01

Manual labeling slowed training and increased errors

Every sentiment and intent model began with thousands of consumer messages labeled by hand in a spreadsheet. The work was slow, physically repetitive, and error-prone. Misclicks and inconsistent judgments introduced noise that surfaced later in model quality.

“After annotating for a while... I get tired and I end up misclicking a lot.”

— Data Annotation Specialist
02

Hotkeys increased speed; confirmations protected quality

Interviews with annotators and benchmarks of Prodigy and Label Studio produced three rules: keep hands on the keyboard, make the hierarchy calm enough for hour four of a shift, and catch accidental labels before they entered the training set.

Annotation tool interface dashboard
An early pass at the dashboard — the day's tasks, results, and benchmarks in one place.
Annotation tool interface with controls beneath the conversation
A layout that puts the classification controls under the conversation, not beside it.
Annotation tool interface for tagging text within a message
A variant for annotating a phrase inside a message, not just the message as a whole.
Hotkeys carry the whole flow — select the text, tag it, move on.
03

Training cycles moved from quarterly to monthly

  • 86% faster than spreadsheets
  • 15% fewer annotation errors
  • Full team adoption within a week
  • Model training cycles: quarterly → monthly

“This is much better to use than working with spreadsheets. Reading the text is much easier, I make fewer mistakes, and I'm much faster at annotation.”

— Lar, Insights Manager
04

Measure agreement, not only throughput

What worked

Removing controls mattered as much as adding shortcuts: fewer choices increased labeling speed while reducing errors.

Opportunities

Inter-annotator agreement — how often two people tag the same message the same way — should have been a day-one metric, not something bolted on after launch.