RelateAnything:实时开放词表关系预测,任意输入即可用

HuggingFace Daily Papers(社区热门论文)·2026-09-11 08:00·5天前
AI 导读

53M 参数的 RelateAnything 可从任意来源的图像区域出发,在推理时以字符串形式给定谓词词表,实时输出带分数的关系预测,速度达 20 ms/帧。它不把物体标签作为输入,跨数据集与零样本场景下平均召回率是最强同规模开放词表方法的 2.3-3.5 倍,并以不到 2% 的参数超过 3B-VLM 场景图模型。模型、语料 RA-4M 与基准 OV-SGG-Bench 均已公开。

HuggingFace Daily Papers(社区热门论文)
47AI 编辑部评分,满分 100

RelateAnything:实时开放词表关系预测,任意输入即可用

2026-09-11 08:00· 5天前
AI 导读

53M 参数的 RelateAnything 可从任意来源的图像区域出发,在推理时以字符串形式给定谓词词表,实时输出带分数的关系预测,速度达 20 ms/帧。它不把物体标签作为输入,跨数据集与零样本场景下平均召回率是最强同规模开放词表方法的 2.3-3.5 倍,并以不到 2% 的参数超过 3B-VLM 场景图模型。模型、语料 RA-4M 与基准 OV-SGG-Bench 均已公开。

Open-vocabulary detection accepts any class list at inference, and promptable segmentation returns regions without class names: the taxonomy has left the model and become an input. Relation prediction has not. Scene-graph models are still trained and evaluated on the 50 or 56 predicates of one annotation style, their relation head conditioned on object labels and so tied to one detector. Three obstacles explain this, none primarily modelling: no relation corpus is both free-text and verified, a label-conditioned architecture cannot accept a vocabulary it was not trained on, and the standard metric rewards agreement with the training corpus, so a larger vocabulary scores as a regression.

We present RelateAnything, a 53M-parameter model taking an image and regions from any source and returning scored relations over a predicate vocabulary supplied at inference as strings. Object labels are never an input, so the region source can change without retraining, and the vocabulary is a bank of text embeddings, not a learned classifier. It runs at 20 ms/frame. Training over 19,103 predicates requires positive-unlabeled supervision and a text encoder that separates antonyms, which contrastive encoders embed at cosine 0.95. To supply the supervision we build RA-4M, 474k images and 4.3M relations over 10,102 free-text predicates, generated against numbered box markers and geometrically verified.

To measure it we build OV-SGG-Bench, six axes scored across datasets that the priors standard recall rewards cannot satisfy. On three cross-dataset benchmarks and a fourth zero-shot, RelateAnything has 2.3-3.5x the mean recall of the strongest open-vocabulary method of comparable scale, margins that survive a real detector, and leads a 3B-VLM scene-graph model on both metrics at under 2% of its parameters. In-domain measurement overstates transfer gains ~5x. Model, corpus and benchmark are public.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org