Case Studies - The work behind the work

A document-generation product ran every long, multi-section generation on a premium frontier model — expensive, and slow at ~39 seconds per document. We treated both how we call the model and which model we call as engineering choices: parallel section generation, model right-sizing, a quality-gated fallback, and response caching.

The result: ~70% lower model cost and ~64% faster generation (39.1 s → 14.2 s), with output quality held at 0.85–1.0 parity against the original premium model — verified by an independent judge, not by the team that built it.

We set out to prove, automatically, that a voice agent keeps a distressed caller safe. A synthetic adversarial caller joins a real voice room, speaks crisis scenarios out loud, and an LLM judge scores the real agent on its exact words. Getting a trustworthy answer meant fixing the test harness more than the product — every “agent failure” turned out to be the harness mis-capturing what the agent actually said.

Once we judged the agent’s complete verbatim transcript from the database instead of a lossy re-transcription, the verdict flipped from “fails” to “passes” on all four crisis categories — and the test got sharp enough to critique therapeutic quality, not just safety.

A self-hosted OLMo-3.1-32B model reached parity with Claude Opus 4.8 on biologics CMC proposal generation — confirmed by a robust multi-judge panel. Retrieval engineering and prompt/enrichment got it to 7:6; a deliberate QLoRA fine-tuning experiment then closed the last gap on the regulatory-judgment workflows RAG couldn’t teach.

The payoff: a frontier-quality proposal engine running fully on-prem, customer-controlled, at a fixed monthly cost.