Gello dataset converter

[tlpss]

I made a converter for the Gello dataset format (pickles containing dicts with all the observations).

If this is of interest, I am willing to contribute it back here.

The current code can be found here. It needs some cleanup and maybe a convenient way to specify the mapping of dict keys in case you have a different number of cameras or other sensors. Wanted to see if there is any interest in this, before I make the effort to clean it up.

[vmayoral]

@tlpss I'd be interested in partnering up on this.

Before setting on a goal though, I'd love to pick your brain on the following:

• your current approach entails converting the teleop commands from Gello into something that lerobot can interpret, from my understanding, a posteriori
• wouldn't it make more sense to integrate Gello abstraction within lerobot so that it can be used directly as a teleoperator. This later direction is the one I'm currently looking at.

[tlpss]

Hi @vmayoral

Thanks for your thoughts.

I indeed was thinking about how to use data collected using the Gello codebase for training policies in Lerobot. (a posteriori integrating gello with lerobot as you described it). This come from our particular setup: We use GELLO arms to teleop robots, based on the original gello codebase. The codebase uses a specific format to store the interactions and I have written code to convert their dataset format into a Lerobot dataset. This dataset converter is my proposed contribution.

However, your idea of directly integrating GELLO arms into the Lerobot codebase so that it can be used as a teleop solution makes a lot of sense.

I think it is related to the question of what scope the Lerobot framework is supposed to have. We use it as a framework for policy training and separate it from the codebase we use to run our robots with and to collect demonstrations. For us (for now), that is the sweet spot in terms of flexibility. However, I can imagine there are use cases where it makes sense to also provide code for controlling the robot and for teleop within the lerobot framework! If that is indeed within scope of the repo (@Cadene @aliberts, I think this is a question for you guys), I surely would be interested to collaborate to add it to the framework! If the maintainers think this is a good idea, I propose we make a separate issue for this to further think about it and track progress?