Move Hypernetworks and Model selection underneath "Generate"

[jangofett890]

At this point there's no reason to hide the options in the settings folder when there's so many different models, and now hypernetworks that one can change out as easily as styles.

Just look at all that space begging to have a drop down for hypernetworks...

[Ehplodor]

For the time being, there are two commits related to hypernetworks but no issue nor discussion (but this one) so dare I ask what is a hypernetwork and what will it be used for in the context of SD ? Thank you in advance.

[hd-x]

the title of the commit was: "added support for hypernetworks (???)"
yes, with 3 question marks. So what are hypernetworks ???

[Buckzor]

I'd very much like to know more about how hypernetworks may be used.

[ogkalu2]

@Buckzor @hd-x @Ehplodor
Novel AI got leaked. It's based on SD but hypertrained on danbooru images. Think Waifu diffusion but much better.

Anyway, a fine tuned model isn't the only change Novel AI made to ensure its quality. The backends connect to hypernetworks as well. This support I'm sure was pretty much entirely for fully utilizing the leaked Novel AI

[bmaltais]

I was able to get the hypernetwork to load... but I do not see any changes for the output produced for the same prompt/seed... How does one make use of the loaded hypernetwork?

[ogkalu2]

@bmaltais
This is what you need to download from the leak

https://cdn.discordapp.com/attachments/1027551135613071380/1027888351757471765/unknown.png

This is what you should do with the files

Rename model.ckpt inside animefull-final-pruned from NAI to final-pruned.ckpt

Place final-pruned ckpt and animevae.pt from the root in stableckpt in stable-diffusion-webui\models\Stable-diffusion

Rename animevae.pt to final-pruned.vae.pt

Make a stable-diffusion-webui\models\hypernetworks folder and place all the .pts from stableckpt\modules\modules from NAI in there

Total disk space used: 11gb
Settings -> Stable Diffusion : Set the model & module and click apply changes

[bmaltais]

Many thanks for the tutorial

[bmaltais]

Observations on using vae and hypernetworks:

I noticed that loading the vae with the ckpt produce a desaturated result. I am not sure what the role of the vae is here. The vae alone does not change the result of the prompt... only the overall saturation and colors (more muted).

On the other hand, the hypernetworks will significantly change the prompt results. I am not sure I can see a relation of the name of the hypernetwork and the prompt changes... but it certainly does something significant. Again... not sure what those hypernetworks actually are...

You can also use the same hypernetworks on current sd1.4 models with significant transformation of the results.

The vae can also be applied to sd1.4 and result in the same color shift observed earlier.

[ogkalu2]

Thanks for the reply. This is very interesting. I don't really know what hypernetworks are either but yes everyone I've talked to has agreed the change is pretty drastic. How would you compare the results with hypernetworks with SD 1.4 to the NAI model. Is one better for normal images ? For anime images ? And what exactly is better ? Is it matching prompts to the output. For instance, trying to get very specific doesn't often work too well, like yellow eyes, red and gold outfit basically here https://www.reddit.com/r/NovelAi/comments/xxnlbz/anime_version_of_nixie_from_1_for_all/?utm_source=share&utm_medium=android_app&utm_name=androidcss&utm_term=1&utm_content=share_button

The normal 1.4 would have a lot of trouble following the prompts to that degree of precision. How does the normal model with hypernetworks fare ?

[bmaltais]

I will hopefully have more time to play around with things... but so far using the following basic prompt: "a beautiful woman" with sd1.4 only and then adding the anime hypernetwork tend to produce more "mature" material amplifying some of the subject characteristics... so to speak... probably a side effect of the type of images it was trained on.

Example from the same seed. I think you will know which is which:

Using other hypernetworks will produce different results. For example, same seed, same prompt with the puny hypernetwork: