Using SDXL in the Webui

[celticsha]

Hi,

When will we be able to use SDXL in the webui please? I have researcher access for 0.9

Thanls

[wzgrx]

SDXL have been made public officially as well now, it is there on stabilityai profile of hugging face:
https://huggingface.co/stabilityai/stable-diffusion-xl-base-0.9
https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-0.9

[celticsha]

And there was me thinking I was special.

[Centurion-Rome]

Currently, it is WORKING in SD.Next (Vlad) :
https://www.reddit.com/r/StableDiffusion/comments/14ub4zn/sdnext_vlad_with_sdxl_09_working_right_now/

[a3nima]

from the update message in command prompt on my last start of A1111 I figure SDXL support is somehow implemented now? How to run it, I've downloaded the model, can it just be run like any oder model?
--> #11757 is this the answer? :)

[jkrauss82]

Can someone for the love of whoever is most dearest to you post a simple instruction where to put the SDXL files and how to run the thing?

I figure from the related PR that you have to use --no-half-vae (would be nice to mention this in the changelog!).

I have tried putting the base safetensors file in the regular models/Stable-diffusion folder but it does not successfully load (actually, it says it does on the command line but it is still the old model in VRAM afterwards).

• Do I need additional files, if yes, which?

• Is this the correct place, if not which is it?

• Do you have to set the VAE coming with the base, if yes where is the correct place to put it?

• Which other settings might prevent the successful loading of the model (CLI args, etc.)?

[jkrauss82]

After doing research and reading through the various threads here and on Reddit mostly, I think the Linux version of Automatic1111 might have an issue with loading the SDXL model due to a memory leak of some kind.

I have not yet been successful in loading the model so far, but I know now this:

Do I need additional files, if yes, which?

No, just the base model file is needed, get it here: https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors

Is this the correct place, if not which is it?

It is the correct place.

Do you have to set the VAE coming with the base, if yes where is the correct place to put it?

Not 100% sure about this one, but the recommended VAE is this one (fixes fp16 problems so --no-half-vae can be omitted as CLI argument): https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae.safetensors

It should be placed next to the base model file, sharing the file name but with added .vae before the .safetensors part.

My shitty low-end system

Core i5 i5-4570S, RTX 3060 12GB, 16GB DDR3, Ubuntu 20.04.6 HWE, nvidia driver 5.30, Cuda 11.8, Python 3.10.11 (main, Apr 5 2023, 14:15:10) [GCC 9.4.0], Version: v1.5.1, Automatic1111 Commit hash: 68f336bd994bed5442ad95bad6b6ad5564a5409a, Startup options: --theme=dark --listen --opt-channelslast --opt-sdp-no-mem-attention

lead me to believe I would lack the necessary amount of memory to load the model. But I have tested with adding zram swap as a last resort and it does not seem that hitting OOM is preventing the model to successfully load, as memory is not completely exhausted during this process.

I am happy to help debugging if someone let me know which additional error reports I should create or logs I should post.

[dhwz]

16GB RAM is just too small to load the model
You've installed TCMalloc on your system? It's more or less mandatory on Linux as there are some mem leak issues without it.
It's also forced to be used automatically (but you have to make sure the libs are installed).

[jkrauss82]

Thanks for the note, not using this right now. I have noticed the warning mentioning TCMalloc is not installed during start up of the webui but have not invested too much thought in it, as for other models it seems to run just fine without it. Will give it a shot and report back here.

EDIT: hasn't helped unfortunately, system now freezes completely when loading the model, before the python process just crashed (Using TCMalloc: libtcmalloc_minimal.so.4).

EDIT2: Got it to run by simply adding swap (never thought I would need that with 16G 🤷). It took 104s for the model to load: Model loaded in 104.4s (create model: 0.6s, apply weights to model: 26.5s, apply channels_last: 1.0s, apply half(): 59.9s, load VAE: 2.7s, move model to device: 12.9s, load textual inversion embeddings: 0.1s, calculate empty prompt: 0.7s).. Inference is okay, VRAM usage peaks at almost 11G during creation of a 1024x1024 image but it has around 1.15it/s. The fp16 fixed VAE didn't work, I had to use --no-half-vae again. Peak RAM+Swap usage during loading the model is around 18-20G from what I have seen in htop.

[jkrauss82]

Anyone reading this and has a similar issue also checkout this PR: #11958

For me it was just enough to get the model load almost completely in RAM, loading time now 20-30s.

[yami-no-tusbas]

well it worked for me simply loading an SDXL model, the gui had a pretty hard time and my ram run out some times... I had to reboot and loading the XL model took about 85% of my 32gigs of ram... then once it was loaded it runs fine.

My cli : set COMMANDLINE_ARGS=--no-half-vae --xformers --disable-safe-unpickle

My rig : 32gb DDR5 and an rtx4090 24Gb of VRAM

[tedbiv]

yes, but i can't use the refiner model... :(

[Moj-X]

SDXL 1.0 seems to be working with --no-half-vae --xformers on my 12Gb RTX 3080 Ti.
Although I have an issue where it has some interlacing type artefacts. Must've done something wrong?

Zoomed in:

[suonnon]

@AlexPlaycorp Any chance for RTX3060ti 8GB VRam?

[Moj-X]

@AlexPlaycorp Any chance for RTX3060ti 8GB VRam?

I believe it's supposed to work. But I'm not sure, sorry.

[GuohongLi]

where to put the SDXL files?

[freecoderwaifu]

Definitely RAM hungry, had to temp downgrade from 64GB of RAM to 16GB and loading it takes way too long since it spills over quite a lot to the pagefile. About 15 min to load if not longer, with heavy system load. Even this PR doesn't help #11958. The model itself works fine once loaded, haven't tried the refiner due to the same RAM hungry issue.

There might also be an issue with Disable memmapping for loading .safetensors files. Having it enabled the model never loaded, or rather took what feels even longer than with it disabled, disabling it made the model load but still took ages.

[dhwz]

That's odd that setting does fix slow loading because of memmapping on Windows. It shouldn't become slower but much faster.
But I recommend running WebUI in WSL2 on Windows.

[freecoderwaifu]

Yeah it does help greatly with all other safetensors checkpoints, I'm not sure if it's an SDXL specific issue but it's still slow loading even with it enabled, mostly due to being RAM intensive.

[mykeehu]

Which model should I use: the base or the refiner version?

[dill-shower]

Base

#11757 (comment)

[alansley]

According to: https://stable-diffusion-art.com/sdxl-model/ - "Sept 6, 2023: AUTOMATIC1111 WebUI supports refiner pipeline starting v1.6.0."

[MrKuenning]

I am getting horrible performance on a 10GB 3080.

4 Minutes for a 512x512.
12 Minutes for a 1024x1024

I tried with
--xformers or --opt-sdp-attention

I added --skip-torch-cuda-test

I did a fresh install of the webui with no extensions.

It says its using 14GB/10GB of my VRAM.

[dhwz]

@MrKuenning you're missing --medvram option

[pcunwa]

Try adding --medvram to the command line argument.
On my PC I was able to output a 1024x1024 image in 52 seconds. (R5 5600, DDR4 32GBx2, 3060Ti 8GB GDDR6)

settings:
1024x1024, DPM++ 2M Karras, 20 steps, Batch size 1
commandline args:
--medvram --opt-channelslast --upcast-sampling --no-half-vae --opt-sdp-attention

[pcunwa]

I used sdxl-vae-fp16-fix and turned off --no-half-vae, which reduced the time to 25 seconds.

[suonnon]

I just tried SDXL 1.0 with 3060ti 8GB vram. It took around 20-40s for 1024x1024
I used: --xformers --no-half-vae --medvram and Nvidia Game Driver v528.49

[MrKuenning]

I thought --no-half-vae forced you to use full VAE and thus way more VRAM.

I ran several tests generating a 1024x1024 image using a 1.5 model and SDXL for each argument.

Using my normal Arguments
--xformers --opt-sdp-attention --enable-insecure-extension-access --disable-safe-unpickle

1.5 = 25s
SDXL = 5:50

--xformers --no-half-vae --medvram

1.5 = 1:30
SDXL = 12:45

--medvram --opt-channelslast --upcast-sampling --no-half-vae --opt-sdp-attention

1.5 = 45s
SDXL = 7:35

RTX 3080 FE 10GB
Driver 536.67
version: [v1.5.1] •  python: 3.10.6  •  torch: 2.0.0+cu118  •  xformers: 0.0.20  •  gradio: 3.32.0  •  checkpoint: [31e35c80fc]

[dhwz]

I thought --no-half-vae forced you to use full VAE and thus way more VRAM.

correct thats why using the fp16 VAE is recommended: https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

But why is SDXL so slow on your device?

[GuruVirus]

Probably missing cudNN DLL replacements or xformers not working properly. Also --xformers and --opt-sdp-attention are exclusive from one another, so try one or the other not both. SDP is faster. Xformers is better for VRAM.

[GuruVirus]

Base works fine with a 4090 (~10 sec inference) and 64GB RAM (~23GB in use without any extensions/pipelines and browser open). Is there a way to enable refiner yet (without an extension)?