A radiology report can already answer a clinical question, so it is hard to tell whether a vision-language model also uses the image. ModaLens, a paired image-swap audit, measures how report availability changes image sensitivity: MedGemma-27B on 3,199 paired MIMIC-CXR cases from 293 patients, all 14 questions per case (13 finding-specific and one composite), each image replaced by one from another study, usually of the same patient, with question and report fixed. Under an explicit answer instruction, the model's generated answer changes on 4.26 percent of trials with the report and 20.94 percent without it, a paired increase of 16.7 points (patient-clustered 95 percent CI 15.6 to 17.7), so report availability reduces image-swap sensitivity under this protocol; the original prompt with a lowercase first-token readout gives 4.70 percent against 17.07 percent, and substitutions also move continuous answer scores where the binary prediction does not change. The labels are derived from reports, which limits conclusions about visual correctness; the direction replicates in two further model lineages. Code, the exact prompts and a run record for every number are at https://github.com/criticaldata/MODALENS.
ModaLens 论文提出图像替换审计法测量医疗 VLM 的影像敏感性
AI 导读
ModaLens 论文提出一种成对图像替换审计方法,测量放射报告存在与否如何影响医疗 VLM 对影像的依赖。
HuggingFace Daily Papers(社区热门论文)
52
AI 编辑部评分,满分 100ModaLens 论文提出图像替换审计法测量医疗 VLM 的影像敏感性
ModaLens 论文提出一种成对图像替换审计方法,测量放射报告存在与否如何影响医疗 VLM 对影像的依赖。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org