Responsible Evaluation of AI for Mental Health

Abstract

Although AI shows growing promise for mental-health care, current evaluation approaches remain fragmented and poorly aligned with clinical practice, social context, and first-hand user experience. We introduce an interdisciplinary framework combining clinical soundness, social context, and equity. An analysis of 135 recent computational-linguistics publications identifies recurring limitations, including reliance on generic metrics, limited participation by mental-health professionals, and insufficient attention to safety and equity. We propose a taxonomy of assessment-, intervention-, and information-synthesis-oriented AI support, each with distinct risks and evaluation requirements, and demonstrate the framework through case studies.

Publication
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)

Related