Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load ltx loras trained with finetrainers #6174

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

neph1
Copy link

@neph1 neph1 commented Dec 23, 2024

Hi
I've been playing around with finetuning LTX-Video using finetrainers, and by default the loras won't load in ComfyUI. I noticed the naming of the keys were slightly different, having a "transformer." prepended.

transformer.transformer_blocks.0.attn1.to_k.lora_A.weight

This PR makes them recognized by ComfyUI, but I have no idea whether it's a viable solution, if it applies to other loras than these, or if it should be fixed elsewhere.

@comfyanonymous
Copy link
Owner

This isn't the correct way of doing this.

The correct way is adding an entry like: https://github.com/comfyanonymous/ComfyUI/blob/master/comfy/lora.py#L384

The even more correct way is to convert loras to the ComfyUI format which is the format used when you use the lora save node.

@neph1 neph1 marked this pull request as draft December 23, 2024 08:34
@neph1
Copy link
Author

neph1 commented Dec 23, 2024

Currently this doesn't work. I'll keep working on it, but don't have time right now.

@neph1
Copy link
Author

neph1 commented Dec 25, 2024

This seems to do the trick. Although, again, I won't argue that this is the right way of doing it, or if all LTX loras look like this. Just close it if it's not admissible.

@neph1 neph1 marked this pull request as ready for review December 25, 2024 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants