I've made an Extension "Embedding Merge"!

[aleksusklim]

Here is the repo: https://github.com/klimaleksus/stable-diffusion-webui-embedding-merge

It creates a tab titled "EM" where you can check text tokenization to see characteristics of individual token vectors:

Also it intercepts your txt2img and img2img prompts at runtime, to let you use new special syntax for merging words or phrases together!

For example, writing something like this:

Fantasy realistic photography: a <'hedgehog'*1.1+'tiger'> is riding a <'bicycle'*0.8+'truck'> under the <'moon'+'sun'*0.9> at <'winter'+'summer'*0.55>. Fantastic details, smooth lighting, award-winning masterpiece.

Might give you an image like that:

Full generate info…

Fantasy realistic photography: a <'EM_1'> is riding a <'EM_2'> under the <'EM_3'> at <'EM_4'> . Fantastic details, smooth lighting, award-winning masterpiece.
Negative prompt: cropped, blurry, bad figures, black-white
Steps: 32, Sampler: Euler a, CFG scale: 7, Seed: 2152753684, Size: 640x512, Model hash: c6bbc15e32, Model: sd-v1-5-inpainting, Denoising strength: 0.4, Conditional mask weight: 0.2, EmbeddingMerge: "<'EM_1'>=<'hedgehog'*1.1+'tiger'>, <'EM_2'>=<'bicycle'*0.8+'truck'>, <'EM_3'>=<'moon'+'sun'*0.9>, <'EM_4'>=<'winter'+'summer'*0.55>", Hires upscale: 2, Hires upscaler: Lanczos, AntiBurn: Count=4;Skip=1;

Detailed documentation is available both at the main page of repository, and also inside the extension itself on its tab.

EmbeddingMerge plays nicely with other extensions like Dynamic Prompts and with custom scripts, notably X/Y/Z plot:

Since my approach is heavily dependent on a few WebUI internals, there is high chance that someday this extension will stop working – in that case, drop an Issue on my repo and I'll try to fix things up!

Also it means that it will work properly only on latest WebUI commits, every time.

[w-e-w]

I've added it to the index

[aleksusklim]

Thanks! And what about my previous extension https://github.com/klimaleksus/stable-diffusion-webui-anti-burn ?

#7464

[w-e-w]

can you give me a concise description (like in one sentence) of what it does so I can use it in Wiki
I need to be off now, but I'll add it when I get back

[aleksusklim]

«Smoothing generated images by skipping a few very last steps and averaging together some images before them» is enough I think.

[w-e-w]

done

[aleksusklim]

I've updated the extension, now it supports entering tokens by their indices!

This allows to merge things of different lengths by adding some non-zero padding to the shortest term. But my personal experiments didn't show any strong benefit from using this.

The syntax: (2653 is the index of token "word ", 3306 is "hello " and 1002 is "world ")

<'first term' + 'second',2653>
<'',3306,1002>

Such concatenation has the highest priority since it directly composes the string before any other operations.
Nevertheless, it may help to construct accurate token sequences even consisting of broken Unicode character parts, or just middle-word tokens that are hard to craft otherwise.

[Wovik253]

Please help! I've often used this extension to simplify work in WebUI by combining frequently used tokens into one embedding, but recently the extension stopped working - instead of a normally working embedding in the "Embeddings" folder there is a file _EmbeddingMerge_temp.pt, opening it with WinPAR I see that where there should be data there is NUL NUL NUL NUL.... I am using the fully updated version of WebUI. i find this extension very useful and necessary, and it is very sad that i can't use it anymore. Help!

[aleksusklim]

Works for me: I've just tested the master and sdxl branches of the extension against the current/latest dev branch of WebUI, and everything is working: runtime embeddings, EM tab vector table, and creation of new embeddings there too.

[Wovik253]

I think I found a way to fix the problem. It is necessary to write in webui-user.bat in COMMANDLINE_ARGS line this argument "--disable-safe-unpickle" and everything worked. Thank you all for your attention, I hope that this lifehack will help someone else.

[monk91]

Hi, thanks for your extension, it's really very useful to me, but I have a question. How can I compile this prompt: (pixel art:1.5), pixelated? Once the file is created it is as if the round brackets and the :1.5 disappear from the token

[aleksusklim]

You should do either (<'pixel art'>:1.5), pixelated or save <'pixel art'> as MYPIXELART and then (MYPIXELART:1.5), pixelated

Round brackets control attention, and attention is resolved after EM, after CLIP, and even after BREAK keyword!
EM works at lowest level, operating on tokens from CLIP initial dictionary. All you can do is multiply: <'pixel art'*3> – while that is not the same as attention, it might give similar results.

Otherwise you're out of luck, and generally you cannot simulate any A1111-specific prompt tricks, like [alternating|words] or [replace:schedule:0.5]

P.S.
Any brackets in strings inside a merge expression will mean literal characters as if they were escaped in prompt \(…\), unless they somehow are weirdly names TI embedding themselves.