Fix race condition in preview code. #6124
Open
+25
−11
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the previous preview code, when possible, a non-blocking `to` operation is performed and, immediately after, the output tensor is used to create an image. If this non-blocking operation has not completed, PIL makes a copy of the uninitialized memory to produce an image. Generally, this will either contain zeros, or the result of a previously generated preview. This results in both incorrect output, and wasted computation (unless the memory this output was eventually copied to is reallocated and displayed instead of a future preview). To resolve this, the state of the preview generation is tracked with an event. - The PIL image is created with no copy - The preview image is not sent to from the server until ready - Completion of this event is polled with a reasonably slow frequency - A new preview is not created if a previous preview has not completed
|
This breaks things on non cuda platforms like mps. |
|
Much appreciated. Is |
|
There seems to be torch.xpu.event and torch.mps.event specifically for those platforms. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the current preview code, when possible, a non-blocking
tooperation is performed and, immediately after, the output tensor is used to create an image. If this non-blocking operation has not completed, PIL makes a copy of the uninitialized memory to produce an image. Generally, this will either contain zeros, or the result of a previously generated preview (Sometimes from an entirely different execution). This results in both incorrect output, and wasted computation (unless the memory this output was eventually copied to is reallocated and displayed instead of a future preview).To resolve this, the state of the preview generation is tracked with an event.
On my system (linux, 2.5.0.dev20240805+rocm6.1, 7900 GRE), I'm seeing no difference in the execution time for latent2rgb and minute performance improvement (~5%) for taesd, but note that erroneous previews were more commonly produced when latent2rgb was used.
EDIT: Fix width and height being swapped in frombuffer call. Minor memory optimization by eliminating tensor concatenation.