Smaller Models for Better Document Understanding
At Cyberhaven, we replaced our previous LLM-based approach to document understanding with a small model that makes 83% fewer decision errors at less than 1% of the inference cost. We achieved this by breaking complex tasks into simpler decisions, using uncertainty as a training signal, and heavily optimizing our inference stack. The model produces calibrated decision probabilities while preserving multimodal understanding and generative capabilities.