PSEBench
A controllable, verifiable benchmark for policy-grounded patient-safety event triage against Minnesota's 29 Reportable Adverse Health Events (MN29).
- Verdict space
- Reportable
- Non-Reportable
- Uncertain
- Tasks
- 146 clause cards
- Cases
- 5,074
- Substrate
- MN29 policy · JQ anchors
What PSEBench measures
Patient safety event (PSE) triage determines whether a clinical event is reportable under jurisdiction-specific policy, a high-stakes task usually performed manually by patient-safety experts. PSEBench evaluates whether an LLM can support this workflow through policy-grounded reasoning rather than mere classification: grounding decisions in legal evidence, identifying missing decision-critical facts, and abstaining when facts or policy criteria are ambiguous.
It is the first comprehensive benchmark for PSE reportability triage that jointly evaluates policy-grounded verdicts, legal-evidence grounding, targeted information-seeking over missing facts, and explicit recognition of policy-level uncertainty.
Paper
PSEBench: A Controllable and Verifiable Benchmark for Evaluating LLMs in Patient Safety Event Triage
Han, Young, Strauss, Hughes, Nesbitt, Schueler, Ngufor, Yang, Xue, Yin (Emory · Scale AI · Mayo Clinic · Vanderbilt)
arXiv:2606.05463