Apple 提出 DiscoSign:语篇感知的文本到手语 gloss 翻译框架

Apple Machine Learning Research(RSS)·2026-09-11 08:00·15小时前
AI 导读

Apple 联合东北大学与 Gallaudet 大学提出 DiscoSign,一个基于 LLM 的模块化框架,用于语篇感知的文本到美国手语(ASL)gloss 翻译,处理空间共指消解、问答从句(QACs)和概念-gloss 一致性三类语篇现象。

Apple Machine Learning Research(RSS)
41AI 编辑部评分,满分 100

Apple 提出 DiscoSign:语篇感知的文本到手语 gloss 翻译框架

2026-09-11 08:00· 15小时前
AI 导读

Apple 联合东北大学与 Gallaudet 大学提出 DiscoSign,一个基于 LLM 的模块化框架,用于语篇感知的文本到美国手语(ASL)gloss 翻译,处理空间共指消解、问答从句(QACs)和概念-gloss 一致性三类语篇现象。

Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.

  • ‡ Equal contribution
  • † Northeastern University
  • § Gallaudet University
  • ** Work done while at Apple

来源:Apple Machine Learning Research(RSS)· machinelearning.apple.com