研究:少样本上下文学习在六种模态中趋同涌现

HuggingFace Daily Papers(社区热门论文)·2026-09-12 08:00·4天前
AI 导读

一项跨模态对照研究提出"趋同涌现假说",即少样本 ICL 一旦涌现,其跨模态难度分布具有共性。实验显示,配对映射式 ICL 在语言、基因组、整数序列、时间序列、图像和蛋白质六种模态中均涌现并超越对照基线,其中五种模态的任务级效应呈相关性。结果部分支持该假说,但并非所有模态都成立。

HuggingFace Daily Papers(社区热门论文)
47AI 编辑部评分,满分 100

研究:少样本上下文学习在六种模态中趋同涌现

2026-09-12 08:00· 4天前
AI 导读

一项跨模态对照研究提出"趋同涌现假说",即少样本 ICL 一旦涌现,其跨模态难度分布具有共性。实验显示,配对映射式 ICL 在语言、基因组、整数序列、时间序列、图像和蛋白质六种模态中均涌现并超越对照基线,其中五种模态的任务级效应呈相关性。结果部分支持该假说,但并非所有模态都成立。

Few-shot in-context learning (ICL), the capacity of a model to infer abstract patterns from input-output examples provided in its prompt and apply them to new inputs, has been extensively studied in large language models trained for next-token prediction on human text. Recently, few-shot ICL has been demonstrated in autoregressive genomic models as well. This raises a question: does ICL emerge broadly across domains, and if so, what common structure is shared? To address both, we develop a controlled cross-modality framework that instantiates the same task suite in a variety of modalities to test what we call the Convergent Emergence Hypothesis: the idea that few-shot ICL, when it emerges, shares a common cross-modality difficulty profile - i.e., tasks that benefit from ICL in one modality tend to benefit in others.

We show that paired-mapping ICL emerges across six modalities (language, genome, integer sequences, time series, images, and proteins), surpasses controlled baselines, and has correlated per-task effects across five of them. Together, these results provide support for the Convergent Emergence Hypothesis in some modalities, but not all.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org