PSF-Med: Measuring and Explaining Paraphrase Sensitivity in Medical Vision-Language Models

PSF-Med is a benchmark of 92,856 audited question-paraphrase pairs, built from 26,850 chest X-ray questions across three countries. Rewording a clinically equivalent question flips the diagnosis on 6.4% to 54.7% of binary pairs across six medical Vision-Language Models (VLMs). The release carries questions, paraphrases, and audit verdicts, not images.

Recommended citation: Sadanandan, B., & Behzadan, V. (2026). PSF-Med: Measuring and Explaining Paraphrase Sensitivity in Medical Vision-Language Models. MMFM-BIOMED Workshop, CVPR 2026.
Download Paper