A way to eject the loaded model to save VRAM?

[giteeeeee]

I often play games while having a1111 still in the background. It eats ~6g of my vram while loaded with whichever checkpoint.

So is there a way to eject the loaded model somehow, without killing the process?

[w-e-w]

Settings > Actions > Unload SD checkpoint to RAM

[watzon]

This doesn't appear to exist anymore, but it is still the answer that shows up first in Google. Is there a new way to do this?

[w-e-w]

This doesn't appear to exist anymore

look again it's there


I recall there is some bugs with the system if you have multiple models loaded at once but it still does unload the current model

[randelreiss]

I've tried this feature. It doesn't appear to free up very much GPU RAM - maybe 10%? And then it frequently crashes AUTOMATIC1111 - I don't know if it's crashed on the web front-end side or something related to the GPU card - but I have to quit AUTOMATIC1111 and restart it to recover.

[andreiramani]

unload checkpoint to RAM, it will eats up RAM if we change model frequently. How to unload completely from RAM and VRAM?

[randelreiss]

Ollama framework has a really handy environment and API accessible variable:

OLLAMA_KEEP_ALIVE=[# of seconds] | [xM] | 0

I think it's mostly used for people who want the last loaded chat model to stay loaded longer. But I use it set to zero to keep the GPU VRAM as empty as possible as soon as possible. This is because I have many users that mostly use the GPU for chat and occasionally for Text-to-speech and SD image creation - loading up the GPU VRAM. Unfortunately SDWeb keeps its last model loaded indefinately. It would be great if SDWeb had a similar Keep Alive option to let us decide how long to keep the last model loaded.

[freakontrol]

I have the same problem using SDWeb as OpenWebUI integration.