Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). The StudentBench platform is freely available at https://studentbench.org.
StudentBench 研究显示 AI 辅导在 GRE 学习增益上与人类专家等效
AI 导读
研究团队发布 StudentBench 评估套件与公开平台,基于超过 175,000 条学生-AI 消息和 2,383 名参与者的数据,发现 AI 辅导在 GRE 学习增益上与专家人类辅导统计等效(p = .015),七个 GRE 领域中有五个领域表现最好的 AI 辅导者平均超过人类。
HuggingFace Daily Papers(社区热门论文)
65
AI 编辑部评分,满分 100StudentBench 研究显示 AI 辅导在 GRE 学习增益上与人类专家等效
研究团队发布 StudentBench 评估套件与公开平台,基于超过 175,000 条学生-AI 消息和 2,383 名参与者的数据,发现 AI 辅导在 GRE 学习增益上与专家人类辅导统计等效(p = .015),七个 GRE 领域中有五个领域表现最好的 AI 辅导者平均超过人类。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org