DDS_zeroshot.ipynb - sds loss derivation

[zoharbarzelay]

i liked your DDS paper a lot, thanks for sharing your code!

It would be really great if you could explain the math/derivation of the SDS loss you're using;
specifically, the following two lines:

grad_z = alpha_t * sigma_t * (e_t - eps)
sds_loss = grad_z.clone() * z

or link any relevant references.

Thx a lot!

[RainbowZhuu]

I have the same question
grad_z = (alpha_t ** self.alpha_exp) * (sigma_t ** self.sigma_exp) * (e_t - eps)
sds_loss = grad_z.clone() * z

[Sutongtong233]

you can refer to this: threestudio-project/threestudio#266.
\delta sds_loss / \delta z will direcly be the grad_z (e_t - eps), which can avoid Unet backward required for Chain Rule.