Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add view matrix gradients & optimize cameras #23

Open
ArthurBrussee opened this issue Nov 13, 2024 · 0 comments
Open

Add view matrix gradients & optimize cameras #23

ArthurBrussee opened this issue Nov 13, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@ArthurBrussee
Copy link
Owner

To enable training from rougher poses (eg. ARCore poses or other SLAM systems), it would be great to have caamera gradients.

This would mainly be a matter of patching up project_backwards to output view matrix gradients that match the ones in gSplat, some of this code is in place.

It does come with other complications:

  • Adding in more bindings might push things over the binding limit on Chrome (only 10 buffers...)
  • Keeping cameras as differentiable tensors will require changing the code in quite a few places. Not 100% sure how to keep that clean.

This would also help for future work in #18

@ArthurBrussee ArthurBrussee added the enhancement New feature or request label Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant