Case Studies - The work behind the work

A document-generation product ran every long, multi-section proposal as one serial call to a premium frontier model (Claude Opus 4.8) — accurate, but slow (~160 s) and expensive. Instead of swapping in a cheaper model, we kept Opus 4.8 and re-engineered how we call it: independent section groups generated concurrently, scoped calls with prompt caching on shared context, and partial-failure salvage.

On the same frontier model, per-proposal cost dropped ~52% (≈$0.87 → ≈$0.42) and end-to-end latency ~74% (≈160 s → ≈41 s), with output quality held — verified by an independent judge, not by the team that built it.

We set out to prove, automatically, that a voice agent keeps a distressed caller safe. A synthetic adversarial caller joins a real voice room, speaks crisis scenarios out loud, and an LLM judge scores the real agent on its exact words. Getting a trustworthy answer meant fixing the test harness more than the product — every “agent failure” turned out to be the harness mis-capturing what the agent actually said.

Once we judged the agent’s complete verbatim transcript from the database instead of a lossy re-transcription, the verdict flipped from “fails” to “passes” on all four crisis categories — and the test got sharp enough to critique therapeutic quality, not just safety.

A self-hosted OLMo-3.1-32B model reached parity with Claude Opus 4.8 on biologics CMC proposal generation — confirmed by a robust multi-judge panel. Retrieval engineering and prompt/enrichment got it to 7:6; a deliberate QLoRA fine-tuning experiment then closed the last gap on the regulatory-judgment workflows RAG couldn’t teach.

The payoff: a frontier-quality proposal engine running fully on-prem, customer-controlled, at a fixed monthly cost.