Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data. However, it produces thousands of redundant visual tokens whose cost grows with every view. Existing visual token pruners fall into two families, each limited in the 3D multi-view setting. Learned importance methods rank tokens by attention or encoder features; because redundancy here is fundamentally spatial, they keep near-duplicate tokens from a few prominent regions and leave most of the scene unrepresented. Voxelization methods improve spatial coverage but cannot enforce an exact token budget and saturate as multi-view observations overlap in 3D, capping retention well below the target. We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals. CoVeR selects tokens that collectively cover every region of the scene, and solves the limitations of both families: it enforces an exact per-scene budget, breaks the voxelization saturation plateau, and avoids the near-duplicate selections of learned importance. Extensive experiments show CoVeR outperforms prior SOTAs on all three 3D reasoning benchmarks and generalizes as a plug-and-play module tested across four VLMs. Notably, with only approx8% of visual tokens, it preserves 93.5% of full-token performance, surpassing SOTA by 3.9 percentage points on average across benchmarks.
CoVeR:面向 VLM 多视图 3D 推理的基于覆盖率的 token 剪枝方法
AI 导读
CoVeR 是一种无需训练的确定性视觉 token 选择器,仅利用 token 坐标实现场景全覆盖,以解决多视图 3D 推理中现有剪枝方法的冗余与预算限制问题。在三个 3D 推理基准上,CoVeR 均超越此前 SOTA,并可作为即插即用模块适配四种 VLM。仅保留约 8% 的视觉 token,即可维持全量 token 93.5% 的性能,平均超出 SOTA 3.9 个百分点。
HuggingFace Daily Papers(社区热门论文)
36
AI 编辑部评分,满分 100CoVeR:面向 VLM 多视图 3D 推理的基于覆盖率的 token 剪枝方法
CoVeR 是一种无需训练的确定性视觉 token 选择器,仅利用 token 坐标实现场景全覆盖,以解决多视图 3D 推理中现有剪枝方法的冗余与预算限制问题。在三个 3D 推理基准上,CoVeR 均超越此前 SOTA,并可作为即插即用模块适配四种 VLM。仅保留约 8% 的视觉 token,即可维持全量 token 93.5% 的性能,平均超出 SOTA 3.9 个百分点。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org