Getting horrible results with thiebaud_xl_openpose #2153

dayunbao · 2023-10-01T11:30:28Z

dayunbao
Oct 1, 2023

I just upgraded my AWS EC2 instance type to a g5.xlarge so it can better handle SD XL. The speed up I got was impressive. Anyway, I did two generations to compare the quality of the images when using thiebaud_xl_openpose and when not using it.

When not using it the results are beautiful:

But when I use it, the results are horrible:

Is anyone else getting similar results? I make use of the OpenPose model extensively, but these results make it unusable.

Here are the prompts and settings I used.

I started the server with the following command line arguments: webui.sh --xformers --no-half-vae

Without thiebaud_xl_openpose:

A (humanoid: 1.5) (samurai:1.2) cat (standing:1.3) in a cyberpunk alleyway lit by neon signs
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1844297559, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, Version: v1.6.0-1-g0562456f

With thiebaud_xl_openpose:

A (humanoid: 1.5) (samurai:1.2) cat (swinging:1.3) a sword in a cyberpunk alleyway lit by neon signs
Steps: 50, Sampler: Euler a, CFG scale: 7, Seed: 1844297559, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, ControlNet 0: "Module: dw_openpose_full, Model: thibaud_xl_openpose [c7b9cadd], Weight: 1, Resize Mode: Crop and Resize, Low Vram: True, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced", Version: v1.6.0-1-g0562456f

dayunbao · 2023-10-01T12:10:12Z

dayunbao
Oct 1, 2023
Author

I just realized that I forgot to include --no-half-controlnet. So I just did that and tried again. The results aren't much better:

The prompt is slightly different, and I used a different sampler:

A (humanoid: 1.5) (samurai:1.2) (black: 1.2) cat (standing:1.3) (holding a katana in one hand: 1.3) in a cyberpunk alleyway lit by neon signs
Negative prompt: Extra limbs, extra fingers, deformed hands, low quality,
Steps: 50, Sampler: UniPC, CFG scale: 7, Seed: 3057226458, Size: 1024x1024, Model hash: 31e35c80fc, Model: sd_xl_base_1.0, VAE hash: 63aeecb90f, VAE: sdxl_vae.safetensors, ControlNet 0: "Module: dw_openpose_full, Model: thibaud_xl_openpose [c7b9cadd], Weight: 1, Resize Mode: Crop and Resize, Low Vram: True, Processor Res: 512, Guidance Start: 0, Guidance End: 1, Pixel Perfect: True, Control Mode: Balanced", Version: v1.6.0-1-g0562456f

4 replies

dayunbao Oct 1, 2023
Author

The results with other OpenPose models aren't much better, in fact, they don't seem to even be using the pose:

t2i-adapter_diffusers_xl_openpose:

kohya_controllllite_xl_openpose_anime_v2:

I used the dw_openpose_full preprocessor with each. Should I have used something different?

huchenlei Oct 1, 2023
Collaborator

I think none of the current SDXL controlnet openpose model is trained with hand/face keypoints data. So when you use these models, you should probably either use openpose preprocessor that does not include hand/face or manually remove hand/face using editor.

dayunbao Oct 5, 2023
Author

OK, but there is still something wrong. The only SD XL OpenPose model that consistently recognizes the OpenPose body keypoints is thiebaud_xl_openpose. But that model destroys all the images. It's unusable. So I've been using kohya_controllllite_xl_openpose_anime_v2, but that only seems to recognize the OpenPose body keypoints about 35-40% of the time, assuming it's recognizing them at all, and it's not just luck that the poses of some of the characters match the keypoints.

An example. This is the input:

These are the results:

The output of the logs during image generation:

Oct 05 06:40:41: 2023-10-05 06:40:41,443 - ControlNet - INFO - Loading model from cache: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]
Oct 05 06:40:41: 2023-10-05 06:40:41,447 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:40:41: 2023-10-05 06:40:41,447 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:40:41: 2023-10-05 06:40:41,483 - ControlNet - INFO - ControlNet Hooked - Time = 0.05967235565185547

Here's the results using thiebaud_xl_openpose:

and the API logs:

Oct 05 06:44:40: 2023-10-05 06:44:40,247 - ControlNet - INFO - Loading model: thibaud_xl_openpose [c7b9cadd]
Oct 05 06:44:40: 2023-10-05 06:44:40,336 - ControlNet - INFO - Loaded state_dict from [/home//stable-diffusion-webui/extensions/sd-webui-controlnet/models/thibaud_xl_openpose.safetensors]
Oct 05 06:44:40: 2023-10-05 06:44:40,348 - ControlNet - INFO - controlnet_sdxl_config
Oct 05 06:44:47: 2023-10-05 06:44:47,437 - ControlNet - INFO - ControlNet model thibaud_xl_openpose [c7b9cadd] loaded.
Oct 05 06:44:47: 2023-10-05 06:44:47,509 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:44:47: 2023-10-05 06:44:47,509 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:44:47: 2023-10-05 06:44:47,541 - ControlNet - INFO - ControlNet Hooked - Time = 7.52367377281189

Do I need to change some settings, or is kohya_controllllite_xl_openpose_anime_v2 just not working?

dayunbao Oct 5, 2023
Author

I made a fresh call after restarting the server to make sure everything was being loaded the same way. Looks like it is:

Oct 05 06:57:46: 2023-10-05 06:57:46,315 - ControlNet - INFO - Loading model: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]
Oct 05 06:57:46: 2023-10-05 06:57:46,915 - ControlNet - INFO - Loaded state_dict from [/home/stable-diffusion-webui/extensions/sd-webui-controlnet/models/kohya_controllllite_xl_openpose_anime_v2.safetensors]
Oct 05 06:57:47: 2023-10-05 06:57:47,440 - ControlNet - INFO - ControlNet model kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb] loaded.
Oct 05 06:57:47: 2023-10-05 06:57:47,447 - ControlNet - INFO - Loading preprocessor: none
Oct 05 06:57:47: 2023-10-05 06:57:47,447 - ControlNet - INFO - preprocessor resolution = 512
Oct 05 06:57:47: 2023-10-05 06:57:47,491 - ControlNet - INFO - ControlNet Hooked - Time = 1.3990898132324219

dayunbao · 2023-10-06T06:17:58Z

dayunbao
Oct 6, 2023
Author

After running a very similar generation in both the web UI and in my app via the API, I think there's a bug somewhere in the API.

Web UI settings:

A cute Asian woman standing in a kitchen
Steps: 50, 
Sampler: DPM++ 3M SDE Karras, 
CFG scale: 7, 
Seed: 3178681427, 
Size: 1024x1024, 
Model hash: 31e35c80fc, 
Model: sd_xl_base_1.0, 
VAE hash: 63aeecb90f, 
VAE: sdxl_vae.safetensors, 

ControlNet 0: 
    "Module: openpose, 
    Model: kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb], 
    Weight: 1, 
    Resize Mode: Crop and Resize, 
    Low Vram: False, 
    Processor Res: 512, 
    Guidance Start: 0, 
    Guidance End: 1, 
    Pixel Perfect: False, 
    Control Mode: Balanced"

Version: v1.6.0-1-g0562456f

Request body:

{
    "prompt": "A cute Asian woman standing in a kitchen",
    "negative_prompt": "",
    "cfg_scale": 7,
    "sampler_name": "DPM++ 3M SDE Karras",
    "batch_size": 8,
    "seed": 0,
    "steps": 50,
    "alwayson_scripts": {
        "controlnet": {
            "args": [
                {
                    "input_image": "a base64 string",
                    "enabled": true,
                    "lowvram": false,
                    "pixel_perfect": false,
                    "module": "openpose",
                    "model": "kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]",
                    "processor_res": 512,
                    "threshold_a": 64,
                    "threshold_b": 64,
                    "weight": 1,
                    "guidance_start": 0,
                    "guidance_end": 1,
                    "control_mode": 0,
                    "resize_mode": 1
                }
            ]
        }
    }
}

Results:

0 replies

dayunbao · 2023-10-06T06:43:25Z

dayunbao
Oct 6, 2023
Author

Another test, since I realized I wasn't passing the image width and height when calling txt2img:

Request body:

{
    "prompt": "A cute Asian woman standing in a kitchen",
    "negative_prompt": "",
    "cfg_scale": 7,
    "sampler_name": "DPM++ 3M SDE Karras",
    "batch_size": 8,
    "seed": 0,
    "height": 1024,
    "width": 1024,
    "steps": 50,
    "alwayson_scripts": {
        "controlnet": {
            "args": [
                {
                    "input_image": "",
                    "enabled": true,
                    "lowvram": false,
                    "pixel_perfect": false,
                    "module": "openpose",
                    "model": "kohya_controllllite_xl_openpose_anime_v2 [b0fa10bb]",
                    "processor_res": 512,
                    "threshold_a": 64,
                    "threshold_b": 64,
                    "weight": 1,
                    "guidance_start": 0,
                    "guidance_end": 1,
                    "control_mode": 0,
                    "resize_mode": 1
                }
            ]
        }
    }
}

I am passing the input_image, by the way. I just didn't want to paste a huge base64 encoded string.

I used the same OpenPose keypoints:

But it still seems that kohya_controllllite_xl_openpose_anime_v2 is ignoring the keypoints:

0 replies

wandrzej · 2023-10-07T17:09:15Z

wandrzej
Oct 7, 2023

overall most if not all openpose controlnets for SDXL are bad. They either ignore the pose, or hit the quality, or both.
afaik there was a half decent one from T2I, but still not even close to what one could get with SD1.5 models.

2 replies

dayunbao Oct 8, 2023
Author

I'll try the T2I model some more, thanks.

My co-founders and I were already thinking about on training our own OpenPose model for ControlNet. Now it looks like we'll definitely have to,

dayunbao Oct 8, 2023
Author

Is this just a problem when using Automatic1111's web UI? Does it work any better in ComfyUI?

dayunbao · 2023-10-08T07:40:49Z

dayunbao
Oct 8, 2023
Author

There is a new OpnePose XL model available from the maker of thiebaud_xl_openpose:

https://huggingface.co/thibaud/controlnet-openpose-sdxl-1.0/tree/main

Maybe this one would work better? @lllyasviel do you think we could get support added for this model?

4 replies

dayunbao Oct 9, 2023
Author

@lllyasviel I dropped that model into the appropriate folder, and tried using it since the yaml files aren't used any more. The API log said it was loaded, but it didn't seem to work. The images didn't respect the OpenPose keypoints at all. I don't know if there is any special configuration that needs to be done before that model can be used, though.

dedkamaroz Dec 31, 2023

I'm just wondering if you've had any updates about this issue?

dayunbao Dec 31, 2023
Author

No, unfortunately. Unless someone has released new ControlNet OpenPose models for SD XL, we're all borked. The problem seems to lie with the poorly trained models, not ControlNet or this extension. So I'll close this. If my startup is able to get funding, I'm planning on setting aside money specifically to train ControlNet OpenPose models.

sdbds Jan 2, 2024
Collaborator

https://huggingface.co/bdsqlsz/qinglong_controlnet-lllite/
you can test my openpose model.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting horrible results with thiebaud_xl_openpose #2153

{{title}}

Replies: 5 comments 10 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Getting horrible results with thiebaud_xl_openpose #2153

dayunbao Oct 1, 2023

Replies: 5 comments · 10 replies

dayunbao Oct 1, 2023 Author

dayunbao Oct 1, 2023 Author

huchenlei Oct 1, 2023 Collaborator

dayunbao Oct 5, 2023 Author

dayunbao Oct 5, 2023 Author

dayunbao Oct 6, 2023 Author

dayunbao Oct 6, 2023 Author

wandrzej Oct 7, 2023

dayunbao Oct 8, 2023 Author

dayunbao Oct 8, 2023 Author

dayunbao Oct 8, 2023 Author

dayunbao Oct 9, 2023 Author

dedkamaroz Dec 31, 2023

dayunbao Dec 31, 2023 Author

sdbds Jan 2, 2024 Collaborator

dayunbao
Oct 1, 2023

Replies: 5 comments 10 replies

dayunbao
Oct 1, 2023
Author

dayunbao Oct 1, 2023
Author

huchenlei Oct 1, 2023
Collaborator

dayunbao Oct 5, 2023
Author

dayunbao Oct 5, 2023
Author

dayunbao
Oct 6, 2023
Author

dayunbao
Oct 6, 2023
Author

wandrzej
Oct 7, 2023

dayunbao Oct 8, 2023
Author

dayunbao Oct 8, 2023
Author

dayunbao
Oct 8, 2023
Author

dayunbao Oct 9, 2023
Author

dayunbao Dec 31, 2023
Author

sdbds Jan 2, 2024
Collaborator