SDXL training implementation progress?

[OWKenobi]

So I would like to collect any progress on SDXL training progress for:

• Loras
• Hypernetworks
• Embeddings

does one of those work for you already? What is minimum ram/vram requirements?

For example: training hypernetworks doesn't work yet: there is an error in the generative-models encoder:

  [...]/repositories/generative-models/sgm/modules/encoders/modules.py", line 141, in forward   emb_out = embedder(batch[embedder.input_key])
TypeError: list indices must be integers or slices, not str

Also Dreambooth doesn't work; it tries to load a v1-file and fails converting the unet

Trying to load: [...]/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/../configs/v1-training-default.yaml
Converting unet...
Exception setting up output: Error(s) in loading state_dict for UNet2DConditionModel:
size mismatch for conv_in.weight: copying a param with shape torch.Size([320, 9, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).

[AUTOMATIC1111]

Maintaining those training scripts is too exhausting of a process for me. I will not work on it and likely will do an update in some future version that removes the tab in favor of some extension.

Everyone trains using kohya's scripts. Kohya's trained SD XL networks can be used in webui.

[FurkanGozukara]

I made the first Kohya LoRA training video

working on a auto1111 video to show how to use

45.) Local - PC - Free - RunPod

First Ever SDXL Training With Kohya LoRA - Stable Diffusion XL Training Will Replace Older Models

[OWKenobi]

Thanks for this tip - I will watch this video later! <3

[OWKenobi]

Thanks for your explanation. Maybe we can still clarify the process a bit; so what resource are we exactly talking about?

https://github.com/ddPn08/kohya-sd-scripts-webui

This one even works as an extension for auto1111, but it doesn't have options for SD XL yet. You might consider adding it to the available extensions list.

https://github.com/bmaltais/kohya_ss

This one looks promising, is also gradio based, and can do XL, but it doesn't mention being an extension.

Interestingly, I cannot find mentionings of hypernetworks in both projects. Have they gone out of fashion?

[catboxanon]

Both of those rely on this repo and is the one auto is referring to: https://github.com/kohya-ss/sd-scripts

Training hypernetworks is also possible, it's just not done much anymore since it's gone "out of fashion" as you mention (it's a very naive approach to finetuning, in that it requires training another separate network from scratch). There's no official write-up either because all info related to it comes from the NovelAI leak.
https://github.com/kohya-ss/sd-scripts/blob/main/finetune/hypernetwork_nai.py

[kuzman123]

sorry @catboxanon but your comment is naive, cause it is obvious you don't know what you can achieve with HypperNets. I'm back to SD 1.5\2.1 only for HN. But i like the fact that training on SDXL is possible.