Chelsea Finn· @chelseabfinn · X·· 2 小时前AI 评分34
AI 导读
生成式价值分类器是一种新的Q函数参数化方式，不需要动作条件化。 我们认为它们或许能与大型预训练策略优雅地结合，并从更丰富的输出信号中更好地学习和泛化。 论文：https://alexanderswerdlow.github.io/gvc/
正文 · 原文
Generative value classifiers are a new parametrization of Q-functions that don't require action conditioning.
We think these may combine gracefully with large, pre-trained policies and learn/generalize better from a richer output signal.
In language models, turning a generative model into a critic is simple: pass in the output and prompt for a score. Robot policies can't do this - they output physical actions & there's no built-in vocabulary to predict scores. So when it comes to RL on robot policies, we typically train a separate critic from sparse rewards - blind to everything the policy learned from the actions in the pretraining data. We think this has it backwards - can a critic learn from those actions the way the policy does? (1/n)在 X 查看被引用的帖子
来源：Chelsea Finn · x.com