[Feature] Add a pause button

[moreAImore]

Feature Idea

Problem: when I launch 10 tasks on ComfyUI, I only have a choice to let them run or stop them
If I want to keep the 10 task in the background without them running, I need to shut down everything.
I still need to keep the remaining tasks without losing them, but yet I still need to pause comfy and potentially liberating a portion of the GPU.
The idea is to have a true pause that let you keep the state of comfy with its remaning taks, but delay their execution until you unpause. In the meantime the pause can alllow you to liberate the memory gpu etc.
That or (#6163) as alternative.

Existing Solutions

No response

Other

No response

[Mooshieblob1]

The implementation of a true pause system would face several significant technical challenges:

1. PyTorch/GPU Memory Management:

• GPU memory handling in PyTorch doesn't have a native suspend state
• Unloading models while preserving state would require complex memory management
• Risk of memory fragmentation when releasing/reacquiring GPU resources

2. Queue State Preservation:

• Current queue system is designed for sequential execution
• Would need to implement serialization for:
  • Full workflow state
  • Intermediate results
  • Model states
  • Random seeds and other execution parameters
• Storage and restoration of partial execution states

3. Resource Management Complexity:

• Need to track and properly release all GPU resources
• Handle cleanup of temporary buffers and intermediate results
• Manage VRAM allocation/deallocation safely
• Ensure proper resource restoration on resume

4. Implementation Risks:

• Memory leaks from incomplete resource tracking
• State inconsistency after pause/resume cycles
• Performance impact from serialization/deserialization
• Increased disk I/O from state storage
• Potential race conditions in the execution pipeline

[moreAImore]

The implementation of a true pause system would face several significant technical challenges:

1. PyTorch/GPU Memory Management:

• GPU memory handling in PyTorch doesn't have a native suspend state
• Unloading models while preserving state would require complex memory management
• Risk of memory fragmentation when releasing/reacquiring GPU resources

2. Queue State Preservation:

• Current queue system is designed for sequential execution

• Would need to implement serialization for:

  • Full workflow state
  • Intermediate results
  • Model states
  • Random seeds and other execution parameters

• Storage and restoration of partial execution states

3. Resource Management Complexity:

• Need to track and properly release all GPU resources
• Handle cleanup of temporary buffers and intermediate results
• Manage VRAM allocation/deallocation safely
• Ensure proper resource restoration on resume

4. Implementation Risks:

• Memory leaks from incomplete resource tracking
• State inconsistency after pause/resume cycles
• Performance impact from serialization/deserialization
• Increased disk I/O from state storage
• Potential race conditions in the execution pipeline

Interesting.
So instead of facing all of these, would it possible to simply save all the remaining tasks into a bucket that can be loaded back? (All tasks saved into api calls sort of, and user can retrieve them)

This will allow some kind pause,
because you can save the state (at least remaining tasks), shut down the system, then come back later when you RERUN comfy.
See issue 6163.

[patientx]

There is a save-load queues custom node (can't remember the name atm), it gives error when installed but works great, say you added a bunch of work on the queue but want to stop and want to continue where you left of (from the very start of current running workflow) you save the queue. close comfy after you open it again when you "load" the queue you saved it immediately starts doing it one by one. Not perfect but really helps when you keep adding stuff and have to log off etc. something like it officially would be great. And for "Pausing" on windows when you select anything on the cmdline window this pauses everything comfy is doing, press space again and it continues the work. This is good when you want to use your gpu for a quick task outside comfy. Of course there are times this cannot go back to work correctly and gives error. Most of the time works ok I guess.