Could we add feature to extract prompt from the current canvas?

[kiwaygo]

As text prompt can be thought of as the primary "control "for stable diffusion, features like CLIP interrogation, wd14 tagger, etc. could be integrated by adding a "star" button next to the prompt textbox, consistent with ControlNet UI.

Providing a way to hook up these vision models for prompt generation into Krita could be very helpful:

1. Enables extracting contents from existing image for generation. Enables content variation via editing the prompt.
2. By prompt extraction then manually narrowing the scope of prompt to be more focused, we could refine the selected parts of image with higher confidence.
3. It prevents writer's block for writing prompt. Maybe it's just me, but when I paint on canvas, having to describe my work from time to time is pretty bothering. Behaviorally, I often ended up with an empty prompt and rely on ControlNet excessively, taking a bet, which is less than ideal.
4. It makes the UI friendlier for, and future-proofs support for models with better natural language comprehension, like Flux. When writing prompt becomes more sophisticated, the tooling and the UI hook would be there ready for integration.

Thoughts?

[fxlab]

I agree, I feed the sketch to the ChatGPT to get the prompt and then amend it as needed. Your proposal would skip that step.

[GeminiSquishGames]

Changing the prompt when detailing is pretty essential too. It also could make regional text from a selection or layer. That's what I would like to have or something to name region prompts after the user splits the image into regional layers, attaching the layers to the right generated prompt. I think it would be within the same feature. It takes a bit of time manually and could use automation with interrogation. Letting one use the layer name string/text as a regional prompt were ideas I was going to try to hack in for my use, but a full feature with interrogation would be way better.