Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support more LoRA models (Flux and SD3.x) #520

Merged
merged 12 commits into from
Dec 28, 2024
Merged

Support more LoRA models (Flux and SD3.x) #520

merged 12 commits into from
Dec 28, 2024

Conversation

stduhpf
Copy link
Contributor

@stduhpf stduhpf commented Dec 11, 2024 •
edited
Loading

Needs more testing.

Seems to work on CPU and Vulkan (unquantized) backends, on my machine at least.

Please do some testing an let me know what works and what doesn't.

Should fix #370 and fix #483

@Green-Sky
Copy link
Contributor

Nice, good job @stduhpf , you have been coding like a machine :)

@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 11, 2024

Nice, good job @stduhpf , you have been coding like a machine :)

Thanks, I'll probably get bored out of it eventually, but so far so good.

@stduhpf stduhpf changed the title Support Diffusers 2 ("A/B") LOrA models (Flux) Support Diffusers 2 ("A/B") LOrA models (Flux and SD3.x) Dec 11, 2024
@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 11, 2024

I was testing SD3 support with https://huggingface.co/ByteDance/Hyper-SD/tree/main?show_file_info=Hyper-SD3-8steps-CFG-lora.safetensors. I can't wrap my head around what's going on with pos_embed, the sizes don't match at all the full model...
Well it seems to work without it, but I would like to understand still.

@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 11, 2024 •
edited
Loading

Welp, I've already found a SD3 Lora that still doesn't work with these changes.
I think I'll need to refactor the whole thing

@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 13, 2024

Ok I think Lora compatibility just got a lot better, but I still found some loras with very weird naming conventions that I don't quite understand so far.

@stduhpf stduhpf marked this pull request as ready for review December 13, 2024 01:05
@stduhpf stduhpf changed the title Support Diffusers 2 ("A/B") LOrA models (Flux and SD3.x) Support Diffusers 2 ("A/B") LoRA models (Flux and SD3.x) Dec 13, 2024
@stduhpf stduhpf changed the title Support Diffusers 2 ("A/B") LoRA models (Flux and SD3.x) Support more LoRA models (Flux and SD3.x) Dec 13, 2024
@stduhpf stduhpf marked this pull request as draft December 13, 2024 19:02
@stduhpf
Copy link
Contributor Author

stduhpf commented Dec 13, 2024

I think this broke previously working LoRAs

@stduhpf stduhpf marked this pull request as ready for review December 13, 2024 19:12
@leejet leejet merged commit cc92a6a into leejet:master Dec 28, 2024
9 checks passed
@leejet
Copy link
Owner

leejet commented Dec 28, 2024

Thank you for your contribution.

stduhpf added a commit to stduhpf/stable-diffusion.cpp that referenced this pull request Dec 28, 2024
@stduhpf
feat: support more LoRA models (leejet#520)
e157c0c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

stable diffusion cpp and hyper.sd sd3 Some Flux loras can't be applied
3 participants
@stduhpf @Green-Sky @leejet