"Prompt steerability" could end up being one of the most economically important properties of a robot foundation model.
simply because, it determines whether a new task requires new training.
This S1 demo (Skild AI's new foundation model that learns from 1 example) is a good show of that.
The kitchen stays roughly the same, but the video instruction changes each time. S1 responds with different action sequences for each requested end state.
All these means, give the model 1 video demonstration, keep the weights fixed, and change the behavior through the prompt.
"Does S1 exhibit physical prompt steerability: different prompts induce distinct behavior in the same environment?" Yes! Watch S1 follow 4 different video promp...