A cheaper model needs calibrated uncertainty before it can escalate safely
Hightouch tested a small decision model that sends uncertain identity-matching judgments to Claude Opus. On its easier synthetic benchmark, the hybrid reached 98.8% accuracy at $0.88 per thousand identities, versus Opus’s 99.0% at $3.86. On harder cases, the hybrid’s cost rose to $2.94 against $5.39: the headline fourfold saving did not carry over unchanged.
Hightouch · Nov 4, 2026
Why it matters
Routing uncertain cases can save money only when confidence scores help distinguish easy decisions from likely mistakes.
More context
The initial benchmark contained 1,000 synthetic identities, evenly split between noisy records for one person and incorrect merges of multiple people. Jev escalated cases in a chosen confidence range. Before copying the approach, measure calibration, error costs and routing thresholds on a held-out sample from your own workload.
Keep in mind
These are the company’s task-specific measurements, not a general model ranking or current pricing guarantee; synthetic balanced data differs from production error prevalence.
Original read