Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] The class MultiOneHot(OneHot) uses the "to_numpy" method from the OneHot class which do not support multionehot vectors. #2633

Open
morales0021 opened this issue Dec 5, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@morales0021
Copy link

Describe the bug

The class MultiOneHot(OneHot) defined at torchrl\data\tensor_specs.py uses the "to_numpy" method from the OneHot class.
The method "to_numpy" from the OneHot class do not support translating multione hot vectors. Therefore an specific to_numpy method should be defined for the MultiOneHot(OneHot) class.

To Reproduce

import torch
from torchrl.data.tensor_specs import MultiOneHot, OneHot

val = torch.Tensor([0,0,1,0,0,0,0,0,0,0,0,1,0,0])
ts = MultiOneHot((11,3))
val = ts.to_numpy(val)

#The value of val is a numpy array with the single element of 2 rather than being [2,0]
print(type(val)) # <class 'numpy.ndarray'>
print(val) # 2

Expected behavior

In the previous code, is expected to obtain a numpy array with values [2,0].
In other words the translation to a numpy should provide a numpy array that matches the number of elements in the tuple provided to the nvec argument of the MultiOneHot class.

System info

Describe the characteristic of your environment:

  • Library installed using pip
  • Python version 3.10.14
  • TorchRL 0.6.0
import torchrl, numpy, sys
print(torchrl.__version__, numpy.__version__, sys.version, sys.platform)

Additional context

Is important to fix this issue because the gym environnements that uses the space MultiDiscrete are translated to MultiOneHot in torchrl (when using the GymWrapper). At some point when interacting with a Gym Environnement, the MultiOneHot needs to be translated back to a sample of the MultiDiscrete space, which needs to be translated correctly.

Reason and Possible fixes

To fix the bug when can use the self.to_categorical method of the MultiOneHot class which correctly translates the data to categorical. We only need to transform it to a numpy array. We can implement a to_numpy method specific to the MultiOneHot class (as shown below)

    def to_numpy(self, val: torch.Tensor, safe: bool = None) -> np.ndarray:
        val = self.to_categorical(val, safe)
        val =  val.long().cpu().numpy()
        return val
``

## Checklist

- [x] I have checked that there is no similar issue in the repo (**required**)
- [x] I have read the [documentation](https://github.com/pytorch/rl/tree/main/docs/) (**required**)
- [x] I have provided a minimal working example to reproduce the bug (**required**)
@morales0021 morales0021 added the bug Something isn't working label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants