1/ Can we replicate the success of large scale pre-training --> task specific fine tuning for robotics?
This is hard as robots have different act/obs space, morphology and learning speed!
We introduce MetaMorph🧵👇
Paper: arxiv.org/abs/2203.11931
Code: github.com/agrimgupta92/m
Conversation
Replying to
2/ MetaMorph is based on the insight that robot
morphology is just another modality on which we can condition the output of a Transformer.
We process an arbitrary robot by creating a 1D sequence of tokens corresponding to depth first traversal of its kinematic tree.
1
1
11
3/ Our pre-trained policy zero-shot generalizes to 1000s of variations in dynamics and kinematic parameters and even completely unseen morphologies. This graph below shows zero shot performance 👇
1
2
4/ Pre-trained controller can zero shot generalize to novel task and morphology combinations. Fine tuning our pre-trained controller is upto 3x more sample efficient than training from scratch on novel tasks.
1
4
5/ Finally, we provide a mechanistic explanation of how MetaMorph is able to control 1000s of morphologies.
MetaMorph simplifies the control problem by learning to activate different motor synergies depending on the input morphology!
1
1
4
3
