FirstRead / Choosing the judge
Why Jev, given those constraints?
We asked five models the same user-risk questions. Jev had the highest AUC in this run; the closest alternative was GLM. Speed and cost help explain the choice.
| Model | AUC | Median check |
|---|---|---|
| Jev | 0.992 | 0.50s |
| glm-5.3-flash | 0.976 | 1.56s |
| gemini-3.8-flash | 0.973 | 2.79s |
| deepseek-v4.1-flash | 0.966 | 7.31s |
| gpt-6-luna | 0.933 | 1.26s |
The close comparison
GLM reaches 0.976 AUC, against Jev’s 0.992.
Its median check takes 3.1× as long. Recorded cost estimates are $0.21 versus $0.04 per 1,000 user checks.
Those costs exclude the companion and reply check. Jev’s scores still need policy calibration; the comparison does not establish clinical reliability.
Between Help and Harm (BHH) test split: AUC uses 1,750 self-harm / no-risk messages; latency uses all 2,044 shared messages. Test labels are model-generated. Models use different scoring interfaces; full ROC curves and methodology are in the appendix.