"embeddings" not working

[tcoulon]

Hello, I'm quite new to SD and even more the Automatic1111. I also use ComfyUI, and there I use files called "embeddings", that have a *.pt extension, to generate real looking faces. In ComfyUI they go in a directory names "embeddings", they are called by their name in the prompt and it works well.
Putting the *.pt files in the embeddings directory of Automatic1111 (stable-diffusion-webui\embeddings) does not work. Renaming the *.pt files to *.bin as I read somewhere does not work. Using models\embedding does not work.
I'm quit sure there is a way... if someone knows I'd be happy to be told the way.

Thierry

[Kadigan]

Okay, so you want to convert those into SafeTensors for A41 to pick 'em up. I used the converter on HuggingFace, and it's picking them up again.

Not sure how well they work, but at the very least they're showing up.

_{(replying, because I found this thread via Google when researching the problem on my own, and since I've found a solution...)}

See below discussion

[Kadigan]

So these should not be showing up regardless of them being .pt vs .safetensors?

Because once this conversion step was completed, they started showing up in the Textual Inversions tab.

Is that wrong? Should I assume that the conversion broke something, and I'm liable to see issues?

[Kadigan]

Ah, I think I understand what you mean... Just because they show up for XL now doesn't actually mean they'll do anything for XL, correct?

I was operating under the incorrect assumption that these were universal, and that them not showing up was an issue somewhere - especially since I checked, saw this thread, and assumed that it's indeed an issue with the software.

_{So I suppose I now need to go through the ones I "converted", look them up, see which model version they work for, and remove the "conversions" when they show up when they shouldn't...}

Or do I still have it wrong, they should work for XL now (after this conversion), and it's just my understanding of "why they didn't show up" that was wrong?

[w-e-w]

don't quote me on this
so I might be wrong on some information
and some details I deliberately glossed over over but this is the basic idea

---

IRRC sd1.5 embeddings can technically "half work" on can work on XL
this is because while sd1.5 only use clip_l, XL uses 2 clip, clip_l and clip_g
embeddings has is all about the clip model, if you have a 1.5 TI (cliip_l) it can be used with xl models (bacause of clip_l part)

if I remeber the code correctly in webui we just filter out 1.5 TIs when XL is loaded
so while architecturally wise it can be used we didn't make it usable

---

I can tell you for I have a some working embeddings in .pt / bin in webui

.safetensors .pt and .bin, are just file types, there are just different ways of storing the same information
as long as the file is valid it can be used

I never use the converter you linked, so I can't say if there's any special sauce in the converter
but if it's truly just a converter that does nothing else than convert the file format, it shouldn't make something that didn't work work

I believe webuiI filter embeddings by looking at the dimension of embedding see if it matched expected dimension if it's not then it's excluded

---

for example, hear is 2 embedding

the left one is for xl and so has clip_l and clip_g, the righ is a 1.5 embedding whitch has only has clip_l

when webui loads a 1.5 model it sees the models is expecting a 768 dim embedding, when presented with a 1280 dim embedding dim it gets filter out, and vice vers

you can certainly use a SD1.5 16x768 embedding on just the clip_l part of XL
or only take the 16x768 part of the XL embedding and use it on SD1.5
"if the code is implemented to handle these situations"
but we didn't do this for webu, we just filter it completely out

[w-e-w]

okay I didn't read the title of the embedding converter you linked I thought it was a .pt or .bin to safetensor converter
but it's a SD1.5 to SDXL Embedding Converter

I'm guessing what it does is basically just grabbing
the SD1.5 embedding and rename it to clip_l and
not sure how it came up with the clip_g part

---

should have guessed
they just filled clip_g with all zeros
https://huggingface.co/spaces/FoodDesert/Embedding_Converter/blob/main/app.py#L24

[Kadigan]

should have guessed
they just filled clip_g with all zeros

Lovely. :D

Okay, so that gives me a little bit more understanding about the situation. Thank you for your explanation - if there are any issues with it, at the very least it gives me an overview of what the issues might be. Very useful, that.

Still, I'm a programmer myself (though I work primarily in PHP and webdev, I sometimes write tools and such too), and that has some unfortunate consequences. One is that I'm prone to see possibilities other people won't even think of, because I've seen - or committed - such atrocities myself in the past... :D

For example: if all SD1.5 embeddings were historically invariably .pt or .bin (because, say, .safetensors were introduced alongside XL - who knows?), then splitting them based on the file extension (on that assumption) is trivial. And that's what I kinda expected happened, either intentionally - or as a bug affecting that particular code.

Anyway, enough of my musings. :D Thank you for the explanation once more.