How to get progress data out of controlnet/txt2img like current images for live preview or etas

[MasonBali]

Hello awesome people of the sd webui controlnet world,

I searched the discussions but could not find the question. I was wondering if there is a possibility to send a GET request for "/controlnet/txt2img" and img2img where we can gather progress information while controlnet is doing its magic? I've tried already "/sdapi/v1/progress?skip_current_image=false'" but this only works for text2img and non controlnet processes. Let me know if I'm missing something where I need to put my glasses on.

Thank you for any advice and help in advance!

Cheerio
Emre

[ljleb]

Strange. for me, GET /sdapi/v1/progress works when a POST /controlnet/img2img is in progress.

In the background /controlnet/*2img is just /sdapi/v1/*2img but with p.scripts.alwayson_scripts = [controlnet_script]. /sdapi/v1/progress should have the same behaviour as the original route.

Here's my request: (on the left and top. on the right is the response. a POST /controlnet/img2img is running in the background)

[MasonBali]

Thank you ljlep for the fast reply!
Alrighty! If that works on your end that is good news, that means something in my setup is not correct hah!! Currently I have the controlnet call wrapped in a python call I think that is the problem instead of going direct in js.

I will investigate and report back of updates. In either way it is great to hear that controlnet is using under the hood the sd/sdapi/v1/*2img!

[ljleb]

Is it possible for you to share how you call the API then? It's okay if you don't want to. Maybe I or someone else could be able to help you more if we had more context. (maybe the info to figure it out is all there, but atm I'm not sure how /progress does not work)

For python code using the extension from within the webui (i.e. from other extensions), there is some efforts that are directed at that at the moment. Maybe you would have a better time interacting with the extension if you had access to a stable interface.

It could be multiple things, maybe your repo is not up to date, maybe it's a corner case the repo is not handling and that I could not reproduce.

[MasonBali]

Hello,

thank you for the reply! I changed the python call to a js call as an async and it is running the controlnet. Now in regards of progress data it is not working. It waits for the controlnet process to finish instead while it is running.

Here is an example code:

async function runText2ImgControlNetDIRECT(prompt, initImageFile, outputName, seed, numSamples, width=512, height=512, steps=15) {
  const http = require('http');
  const database = path.join(__dirname, '/../../database');
  const contentDir = path.join(database, 'content/');
  const outputDir = path.join(database, 'output/__out');
  const initImage = path.join(contentDir, initImageFile);
  const outputFile = outputName + ".png";
  const outputPath = path.join(outputDir, outputFile);
  const cfgScale = 7;

  const imageData = fs.readFileSync(initImage);
  const base64Image = imageData.toString('base64');

  prompt = "Keanu Reeves"
  negativePrompt = "ugly limbs"

  const postData = JSON.stringify({
    "prompt": prompt,
    "negative_prompt": negativePrompt,
    "controlnet_input_image": [base64Image],
    "controlnet_module": "depth",
    "controlnet_model": "control_depth-fp16 [400750f6]",
    "seed": seed,
    "sampler_index": "Euler a",
    "batch_size": numSamples,
    "steps": steps,
    "width": width,
    "height": height,
  });

  const options = {
    hostname: '127.0.0.1',
    port: 7860,
    path: '/controlnet/txt2img',
    method: 'POST',
    headers: {
      'Accept': 'application/json',
      'Content-Type': 'application/json',
      'Content-Length': Buffer.byteLength(postData)
    }
  };

  checkProgress();

  return new Promise((resolve, reject) => {
    const req = http.request(options, (res) => {
      let data = '';
      res.on('data', (chunk) => {
        data += chunk;
        console.log(data);
      });

      res.on('end', () => {
        console.log(JSON.parse(data)['images']);
        resolve(JSON.parse(data)['images']);
      });
    });

    req.on('error', (error) => {
      console.error(error);
      reject(error);
    });

    req.write(postData);

    req.end();
  });
}


function checkProgress() {
  const http = require('http');

  const options = {
    host: 'localhost',
    port: 7860,
    path: '/sdapi/v1/progress',
    method: 'GET',
    headers: {
      'Accept': 'application/json',
      'Content-Type': 'application/json',
    }
  };

  const interval = setInterval(() => {
    const req = http.request(options, (res) => {
      let data = '';
      res.on('data', (chunk) => {
        data += chunk;
      });

      res.on('end', () => {
        const progressData = JSON.parse(data);
        console.log(progressData);
        if (progressData['progress'] === 1) {
          clearInterval(interval);
        }
      });
    });

    req.on('data', (chunk) => {
      console.log(chunk);
    });

    req.on('error', (e) => {
      console.error(`problem with request: ${e.message}`);
      clearInterval(interval);
      reject(e);
    });

    req.end();
  }, 1000);
}

I'm calling runText2ImgControlNetDIRECT in another js file like this:

const results = await runText2ImgControlNetDIRECT(prompt, initImage, outputName, seed, 3);
// do stuff with results...

Any idea what might cause issues?
And many thanks for the help, I appreciate it a lot because I'm already eating cookies under the shower in fetal position for days now on this! :D

Cheerio
Emre

[ljleb]

Thanks for the code! I have to look up one or two definitions to understand if the code blocks on requests or not.

In the meantime, can you confirm that you are running this from within the webui and not in a separate client?

[MasonBali]

Many thanks for the support!

Yes I confirm, I am running beforehand in a terminal the webui-user.bat with the --api command and not in a separate client. And my setup is briefly something like this where I am calling in a main.js all the ControlNetDirect code or SD txt2img one and inside the terminal where I have launched the webui-user.bat I see the progress and process every time I send a request but not on the console.log in my browser. Interestingly the progress request checkProgress(); is working when I run it for the normal sd web ui txt2img or img2img or depth2img after I also converted the previous pyshell call to a js one.
I call checkProgress(); similarly to the above code before the return Promise post request when I do that for the normal sd webui txt2img one.
I will also keep investigating and if I find something or anything I will definitely post it here.

Cheerio
Emre

[MasonBali]

-SOLVED-

Alrighty IT WORKS!

I realized after trial and error that the solution was to just git pull the latest updates on the repo!!!! HAHAHA!! I was working with an older version and many things changed here. So a git pull solved the issue. Progress data can be tracked.

This is resolved! So happy right now! :D
Thanks for all the help @ljleb!!

Cheerio
Emre