FORCE_SOFT_EMPTY_CACHE env var to enable cache clearing for AMD GPUs

[bigcat88]

This PR adds support for the FORCE_SOFT_EMPTY_CACHE environment variable to enable cache clearing for AMD GPUs. The logic for cache clearing already exists and is used for NVIDIA GPUs; this change simply extends the same functionality to AMD GPUs when the variable is set.

The feature has been tested for a long time on two AMD video cards, the 7900 GRE and 7900 XTX, using PyTorch versions 2.4.1, 2.5.1, and 2.6.0.dev. In all cases, enabling the cache reset worked without issues.

To maintain stability, the default behavior remains unchanged as this functionality may not be suitable for all AMD GPUs (as mentioned in an existing comment). Users can enable this behavior by setting the FORCE_SOFT_EMPTY_CACHE=1 variable.

[comfyanonymous]

This option is like this by default because clearing cache used to slow things down slightly on AMD. If they fixed it I can just change the behavior.

[bigcat88]

We need someone to check this on not officially supported video cards by ROCM (not AMD 79xx series) and with ZLUDA - judging by the discussions/research here how popular it is...
Or you can change this, and then when issue will arise - revert it back.

[patientx]

We need someone to check this on not officially supported video cards by ROCM (not AMD 79xx series) and with ZLUDA - judging by the discussions/research here how popular it is... Or you can change this, and then when issue will arise - revert it back.

I can test it, I have a rx 6600 .How does it work though ? Does it do this automatically when certain criteria met ? After adding these changes to model management, I activate venv and THEN set it with "FORCE_SOFT_EMPTY_CACHE=1" before starting comfyui , is this right ?

[bigcat88]

yes, run ComfyUI with it:

FORCE_SOFT_EMPTY_CACHE=1 python3 main.py

or just change this line:

def soft_empty_cache(force: bool = FORCE_SOFT_EMPTY_CACHE):

to

def soft_empty_cache(force: bool = True):

and then you can skip setting env var.

[patientx]

yes, run ComfyUI with it:

FORCE_SOFT_EMPTY_CACHE=1 python3 main.py

or just change this line:

def soft_empty_cache(force: bool = FORCE_SOFT_EMPTY_CACHE):

to

def soft_empty_cache(force: bool = True):

and then you can skip setting env var.

what kind of scenarios should this be tested with ? (By the way, I needed to add import os , because of the os.env command at the start.)

[bigcat88]

since I always used Comfy on AMD with this option enabled initially, I can't say exactly what you should pay attention to.

my guess: probably some slowdowns when switching models/workflows, when one model will be unloaded and second loaded. or some complex workflows when "text encoder" will be unloaded to free space for KSampler

[devarped]

Haven't tested it on Comfy, but on dev branch of reForge (which is using Comfy's model management code) force-enabling soft_empty_cache has solved the issue of draconian VRAM usage on my 7600 XT. Before that it would use all of 16 gigs of VRAM when doing something as basic as generating 4 images with any SDXL model, now it sits at ~6 gigs after loading the model and at ~8 when generating with no noticeable slowdowns.