Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong DDIM inversion step #79

Open
Tim3s opened this issue Nov 18, 2023 · 1 comment
Open

wrong DDIM inversion step #79

Tim3s opened this issue Nov 18, 2023 · 1 comment

Comments

@Tim3s
Copy link

Tim3s commented Nov 18, 2023

def next_step(self, model_output: Union[torch.FloatTensor, np.ndarray], timestep: int, sample: Union[torch.FloatTensor, np.ndarray]):
    timestep, next_timestep = min(timestep - self.scheduler.config.num_train_timesteps // self.scheduler.num_inference_steps, 999), timestep
    alpha_prod_t = self.scheduler.alphas_cumprod[timestep] if timestep >= 0 else self.scheduler.final_alpha_cumprod
    alpha_prod_t_next = self.scheduler.alphas_cumprod[next_timestep]
    beta_prod_t = 1 - alpha_prod_t
    next_original_sample = (sample - beta_prod_t ** 0.5 * model_output) / alpha_prod_t ** 0.5
    next_sample_direction = (1 - alpha_prod_t_next) ** 0.5 * model_output
    next_sample = alpha_prod_t_next ** 0.5 * next_original_sample + next_sample_direction
    return next_sample

This is the code for calculating 'next step' in null_text_w_ptp.ipynb, but I think it's different from the equation in your paper.

According to your paper, next step can be calculated as:
$$z_{t+1}=\sqrt{{\alpha_{t+1}}\over \alpha_t}z_t + \left(\sqrt{{1 \over {\alpha_{t+1}}} - 1} - \sqrt{{1 \over \alpha_t} - 1}\right)\cdot \epsilon_\theta (z_t, t, C)$$

Latent is calculated at timestep $t$, and this code calculates latent from timestep $t-20$ to timestep $t$, so this code performs the same as using $\epsilon_\theta (z_t, t+20, C))$ instead of $\epsilon_\theta (z_t, t, C))$.

@Tim3s Tim3s closed this as completed Nov 18, 2023
@Tim3s Tim3s changed the title wrong DDIM inversion wrong DDIM inversion step Nov 18, 2023
@Tim3s Tim3s reopened this Nov 18, 2023
@Hprairie
Copy link

DDIM allows for faster diffusion. It looks like they are taking 1000 diffusion steps total with 50 sampling steps, thus a stride of 20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants