Realism Is Not Enough: Meta-Evaluating Safety Evaluation in Mental Health LLMs
Overview
This project examines static benchmarks and simulated multi-turn conversations, focusing on their assumptions about user behavior and automated judgments of safety. The paper argues that more realistic conversations do not automatically provide stronger evidence of safety: conclusions must reflect what an evaluation measures and validates.
Ongoing work translates these findings into practical guidance for matching safety claims to appropriate evaluation methods and evidence.
Abstract
As large language models are increasingly explored as tools for mental-health support, evaluating whether they can interact safely with psychologically vulnerable users has become an important challenge. Mental-health LLM evaluation has been moving beyond static benchmarks towards simulation-based methods intended to better approximate real user interactions over many turns. However, greater interactional realism should not inherently imply greater validity. Static benchmarks offer experimental control but provide limited evidence about harms that emerge over time, while simulation-based methods introduce new assumptions about synthetic users, conversational trajectories, and LLM judges. We argue that psychological-safety evaluation requires stronger claim–method alignment: evaluation methods should be selected according to the claims they are intended to support, and conclusions about psychological safety should be calibrated to assumptions that have actually been validated.
Working paper here.