We could update to last Torch & Xformers?

[ZeroCool22]

https://pypi.org/project/xformers/

https://pytorch.org/get-started/locally/

Those versions have some speed increase?

[AIEXAAA]

You can upgrade, but be careful about the CUDA version that corresponds to Xformers.

I have installed PyTorch 2.4.0 with CUDA 12.4. On an RTX 4080, SD1.5 is about 15% to 20% faster, and SDXL is about 10% faster.

[ZeroCool22]

You can upgrade, but be careful about the CUDA version that corresponds to Xformers.

I have installed PyTorch 2.4.0 with CUDA 12.4. On an RTX 4080, SD1.5 is about 15% to 20% faster, and SDXL is about 10% faster.

Thx, i will need a little more help from you to not fuck up anything.

In my Win. 10 i have installed:

Now for what i understand AUTO's create his own ENV independent of the PC ENV, so in that ENV there are separate versions for CUDA, Torch, etc...

So should follow this guide: https://krdevnotes.com/update-and-upgrade-torch-and-xformers-on-windows-11-for-automatic1111s-stable-diffusion-webui-2024/

But change the final command to inside (venv) C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\venv\Scripts>:

pip install --upgrade torch torchvision xformers --index-url "https://download.pytorch.org/whl/cu124"

And in case something goes wrong i simply could delete the VENV folder and let it regenerate next time i launch AUTO's right?

EDIT:

Precompiled xFormers for CUDA 12.4 and PyTorch 2.4 Compatibility

facebookresearch/xformers#1079

[solss-bit]

Definitely faster, went from 15 seconds to 13 seconds, but Adetailer face seems broken as a result, it finds literally 100 faces after making the change -- mesh still works. Went back to 2.3.1 for now.

[AIEXAAA]

Definitely faster, went from 15 seconds to 13 seconds, but Adetailer face seems broken as a result, it finds literally 100 faces after making the change -- mesh still works. Went back to 2.3.1 for now.

This will be the correct command to update Xformers?

pip install --upgrade torch==2.3.1 torchvision==0.14.1 xformers==0.0.27.post2 --index-url "https://download.pytorch.org/whl/cu121"

Version 0.0.27 just got published, and is compatible with PT 2.3.1

facebookresearch/xformers#1052

pytorch:

pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

It seems that xformers 0.0.27 does not have dependencies for CUDA 12.4. You need to (assuming you have xformers installed):

pip install --upgrade xformers==0.0.28.dev881

[DvST8x]

Thanks, seeing about a 10% speed improvement on my 4090. (opt-sdp-attention, No xformers).

For anyone else that runs into this or similar error after updating "OSError: [WinError 126] fbgemm.dll" or one of its dependencies"
Sharing the solution I found., A copy of libomp140.x86_64.dll needs to be in C:\Windows\System32

libomp140.x86_64.zip

[ZeroCool22]

Definitely faster, went from 15 seconds to 13 seconds, but Adetailer face seems broken as a result, it finds literally 100 faces after making the change -- mesh still works. Went back to 2.3.1 for now.

This will be the correct command to update Xformers?
pip install --upgrade torch==2.3.1 torchvision==0.14.1 xformers==0.0.27.post2 --index-url "https://download.pytorch.org/whl/cu121"
Version 0.0.27 just got published, and is compatible with PT 2.3.1
facebookresearch/xformers#1052

pytorch:

pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

It seems that xformers 0.0.27 does not have dependencies for CUDA 12.4. You need to (assuming you have xformers installed):

pip install --upgrade xformers==0.0.28.dev881

Awesome, thx!

[ZeroCool22]

Thanks, seeing about a 10% speed improvement on my 4090. (opt-sdp-attention, No xformers).

For anyone else that runs into this or similar error after updating "OSError: [WinError 126] fbgemm.dll" or one of its dependencies" Sharing the solution I found., A copy of libomp140.x86_64.dll needs to be in C:\Windows\System32

libomp140.x86_64.zip

I only see this but i think is just a warning:

venv "C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: v1.10.1
Commit hash: 82a973c04367123ae98bd9abdf80d9eda9b910e2
CUDA 12.4
Launching Web UI with arguments: --xformers --api
C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\flash.py:210: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_fwd")
C:\Users\ZeroCool22\Desktop\AutoSDXL\stable-diffusion-webui\venv\lib\site-packages\xformers\ops\fmha\flash.py:344: FutureWarning: `torch.library.impl_abstract` was renamed to `torch.library.register_fake`. Please use that instead; we will remove `torch.library.impl_abstract` in a future version of PyTorch.
  @torch.library.impl_abstract("xformers_flash::flash_bwd")
CHv1.8.11: Get Custom Model Folder

[camoody1]

Definitely faster, went from 15 seconds to 13 seconds, but Adetailer face seems broken as a result, it finds literally 100 faces after making the change -- mesh still works. Went back to 2.3.1 for now.

This will be the correct command to update Xformers?
pip install --upgrade torch==2.3.1 torchvision==0.14.1 xformers==0.0.27.post2 --index-url "https://download.pytorch.org/whl/cu121"
Version 0.0.27 just got published, and is compatible with PT 2.3.1
facebookresearch/xformers#1052

pytorch:

pip install --upgrade torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

It seems that xformers 0.0.27 does not have dependencies for CUDA 12.4. You need to (assuming you have xformers installed):

pip install --upgrade xformers==0.0.28.dev881

For anyone coming to this thread after Sept 28, 2024, the latest xformers version for pytorch 2.4.1+cu124 is:

pip install --upgrade xformers==0.0.28.post1

[AlexKaleda]

I followed the advice by AIEXAAA and DvST8x and indeed, the generations are faster now! My SD1.5 test case went from 21 s to 19 s. Another major effect that was not mentioned - model loading is 4x faster now!
On the negative side, it randomly caused a crash in KERNELBASE.dll during generation.

[solss-bit]

Yeah, if you use adetailer maybe hold off for a moment: Bing-su/adetailer#687
Guess I was either unlucky, or you got lucky without experiencing the issue?

[ZeroCool22]

Yeah, if you use adetailer maybe hold off for a moment: Bing-su/adetailer#687 Guess I was either unlucky, or you got lucky without experiencing the issue?

I already updated and i don't have any problems with ADETAILER.

2024-08-23.02-40-36.mp4

Nvidia drivers version: 560.81

[solss-bit]

I went from a 2080 to a 3090 and no more issues with 2.4.0. Maybe an issue with that series of GPUs?

[TimmekHW]

На 4060ti прирост c 1.6 - 2 it/s

На 2.1 - 2.4

Прирост 20%!

[TimmekHW]

Забыл сказать, что я обновил cudnn до 9.3.0 (в самой папке stable-diffusion-webui\venv\Lib\site-packages\torch\lib) и pytorch 2.5.0+cu124

[cfg138]

Interesting, today i test torch240+cuda12.4 vs torch212+cuda12.1 (nv4080, only sdp-no-mem, a1111 1.10.1) and old version much faster for me @SDXL 832x1216 - 6-7+sec vs 5sec (cold and normal bake) @SDXL 832x1216 hires x2 1.30+min vs <1min (sometimes cuda12.4 crash with out of memory). I think torch231+cuda12.1 also slower for me, dont sure