FirstRead / Design constraints
What checking every turn demands
For an accessible companion, the safety check must be fast, affordable on every turn, and inspectable when it is wrong.
Latency
The check sits in front of every reply.
Read the user while the companion drafts. Check the reply before releasing it.
Cost
Checking only a sample would leave turns unchecked.
Use a small structured check on every turn; measure its cost separately from generation.
Useful scores
A binary flag gives us little room to tune the policy.
Ask specific questions, then choose cutoffs from measured recall and false alarms.
A score is not a clinical probability. Jev’s mid-range scores run low on this benchmark; we use them to rank evidence and tune cutoffs, rather than reading 0.3 as a 30% chance of harm.