Overly saturated images when chaining images using img2img (and with alternative test)

[dellis23]

I've seen a few people mention this in various discord channels as well as on reddit. I've run into a similar issue.

The overall workflow: take an image generated with txt2img. Put it in img2img along with a controlnet image. Generate a new image. This image appears more saturated than the previous one. Rinse and repeat a few times, and things quickly get out of control.

Here's an example.

First image:

Second image:

Fifth image:

Any further than that and it starts to get ridiculous:

I've started a discussion rather than an enhancement request because I don't know if the functionality is already present to prevent this from happening. Turning on sigma adjustment "fixes" this particular problem, but introduces a new one; eventually, the image is washed out by a color rather than being super saturated:

I've tweaked every knob I can think of to try to fix it (randomness seemed hopeful), but changing anything usually keeps the same problem and introduces a new one.

Not sure who to ping on the discussion. Perhaps @AUTOMATIC1111 @MartinCairnsSQL @elias-sundqvist (hope you don't mind the ping).

[ghost]

How low/high is the denoise for this example?

I never tried to use img2img with controlnet too much, but I think it might be the color correction? Deactivating it is probably not a real solution but you can comment out the line in processing.py that is pretty far on the top "image = blendLayers(image, original_image, BlendType.LUMINOSITY)". This makes the color correction work in a slightly different way. It is in the function apply_color_correction. Tbh, I always have this line commented out, and I also save the img2img image before applying color correction, but use the color corrected image to feed it back into loopback. This way the artifacts from the color correction dont show up on the images, like color banding embossing etc. Short on time right now but you can search images.py for the part where it saves the image and hack it a bit to do this

If doing loopback I recommend using the MSE VAE for SD 1.5 instead of the EMA one. It is smoother

[dellis23]

Are the VAE changes reflected in the API too, or only via the web UI? I'm having trouble making sure I'm getting a consistent repro, since refreshing the UI to make sure the VAE changes take effect causes my prompt to reset (and reloading it from the image browser doesn't fill in the controlnet and img2img alternative test script settings).

From what I can tell, commenting the image = blendLayers(image, original_image, BlendType.LUMINOSITY) seems to do nothing, or at least, the saturation problem is still present. Using MSE VAE along with the commented out line gives me this via the web UI (after two iterations):

Would the color correction be affecting the preview image too? One observation I have is that the preview image gets pretty nuts before outputting a more sane image, and I think I've only seen this effect when doing img2img and the saturation problem happens:

It doesn't happen every time though.

RE: denoise: I was running with override to 1 set. But I've tried at various values and the problem seems present on all of them except for 0.

[MitsuhaKitsune]

I have the same problem, ocurrs with inpainting too