integrate the Hydrus Client API

[CuddleBear92]

Integrate the Hydrus Client API!
having a folder to manage your generated files is messy and gets out of control quite quickly!
With Hydrus you can push new files to an Hydrus client and also send over any tags or prompt the user would use.
This will also work great with Deeper Danbooru!

Easier management of larger library of image generation, don't be limited by the explorer window.
Hydrus is also supported by Windows, Linux and MacOS across the board or even run from source in Python.
The API can push raw files, urls, tags and so on if you like.
Technically if integrated more fully you can also query files on the Hydrus Client to use in img2img and others.

Don't be limited by the simple limitations of filenames and folders, use tags for your seed, prompt, tags, model, settings and more!

https://github.com/hydrusnetwork/hydrus
Docs: https://hydrusnetwork.github.io/hydrus/index.html
API docs: https://hydrusnetwork.github.io/hydrus/client_api.html
https://hydrusnetwork.github.io/hydrus/developer_api.html

Side note, Hydrus can also scrape sites and boorus if you want to download a new imageset for training!

[evanjs]

Thank you for opening this issue 😄
There are a ton of things I want to do re stable diffusion and Hydrus, but getting the discussion going is a great place to start, regardless of what auto wants to do
(Maybe the issue or just this comment would be better served as a discussion, idk)

I'll add a few things I've been thinking about that would be nice to have re webui <-> Hydrus integrations

Note: I don't think that all of these are necessarily "best implemented in the WebUI", but I'll list them here to consolidate my thoughts/before I forget

---

Easier tag/generation parameter importing

Hydrus supports configuring "import folders", which allow you to specify a folder to import at specific intervals. It can also delete files after import. This makes Hydrus a good candidate for long-term storage of content generated via SD, especially at higher volumes.

While you can automatically import images from folders with Hydrus, there are a few quirks that prevent the above from functioning perfectly with webui-generated content:

• Hydrus supports loading tags for any imported image from "sidecar" files (files next to imported image ending in .txt) .
  However, the sidecar file format is <name>.<suffix> rather than <stem>.<suffix>.
  e.g. WebUI outputs 00007-1110204304.txt, while Hydrus would expect 00007-1110204304.png.txt, and so on.
• Hydrus does import EXIF data, but there is currently nothing that parses imported EXIF data to tags, etc.
  What this means, is that an imported image will have no additional information (such as generation parameters) that is easily visible in Hydrus.

Currently, I have to add tags for each image manually. While this is not a problem exclusive to this scenario, it is exacerbated by how much I can easily generate with SD, and quickly gets tiresome.

---

While there is quite a bit more that Hydrus can do, I think that being able to automatically move generated images into Hydrus—with information such as generation parameters—would make the volume of SD content I generate much easier to manage in the long run.

As stated above, Hydrus can already do scheduled/automatic folder imports, but the current end result is essentially just a giant scheduled picture dump without easily visibile/associated tags, which doesn't entirely solve the long-term storage/management problem.

As it stands, Always save all generated images will be the death of me, especially now that #1932 (i.e. Generate Forever) has been merged.

Regardless, thanks for your continued work on this repository, Auto!

(And please pardon the brain vomit 😅)

[CuddleBear92]

Well working outside the API works well and good, but having the API will allow for much more features, like querying files for img2img use or even training data. textual diffusion training on a Hydrus query would be pretty cool.

Sidecar files work but you are limited on the complexity of it all.

[sbstratos79]

For now, I am mass renaming tag files and letting Hydrus import all the images. I was using hydrus-dd for auto tagging. Now that deepdanbooru has been integrated with this repo, I won't have to use hydrus-dd. But integration with Hydrus api would be really nice.

[horribleCodes]

In search of a proper storage system, I've been looking into Hydrus. There isn't any news on this, is there? I want to manually tag as little as possible, and we have all this data in the chunk already, so it would be a lifesaver if there was some way to make this work.

[evanjs]

Officially: Not yet as far as I know.
Unofficially: I patched Hydrus to parse tags from the text files produced by automatic, and set up an import folder for my stable diffusion outputs directory.

This effectively imports and tags my outputs automatically.

I try to stay up to date with official releases, so I'll post anything here if I notice any significant official changes.

PS: Hydrus dev mentioned they're touching up sidecar (txt files next to the images) file parsing this week, so my patch might not be required, we'll see.

[horribleCodes]

Is there some tool that lets me batch extract chunk data into a sidecar that Hydrus can read? I assume the sidecar feature was added not too long ago, because there's a lot of images that don't have one. To make matters worse, when I tried using https://github.com/demibit/stable-toolkit as an alternative, it renamed all of my images, along with the text files to hashes - and they don't match any more.
I looked for an existing tool that could scrape the info in bulk, but there doesn't seem to be anything available. I did find this https://github.com/hughsk/png-chunks-extract however, which should be able to do the job, though I don't know much about JS, so it would take me a while.
Any help would be greatly appreciated.

[BrainDeadZero]

The webui has a setting you can enable to create an accompanying .txt file every time you generate an image.

Create a text file next to every image with generation parameters.

If the image is image.png, the text files would be image.txt. However, Hydrus expects the format to be image.png.txt. This seems like a simple enough fix. Just append .png to the end of the file name. When importing to Hydrus, enable the setting:

try to load tags from neighbouring .txt files

Once imported, the file will have the prompts and settings included as tags. Hydrus sees every newline as its own tag, so you'll have a separate tag for the prompt, negative prompt and the various settings.

Personally, I want to take this a step further and separate all the various settings like sampler and CFG scale into its own tag.

At the moment, I am doing the above manually. There might be a way to use regex to read the settings as individual tags, but I'm not familiar enough with it. Alternatively, maybe there's code in the webui that can be modified so the .txt file formats every setting as newline-separated.

[evanjs]

Yeah, once the metadata is read from the file, I just split the settings line by commas.

Currently my output contains the namespaces:
Prompt
Negative prompt
Positive (for individual tags)
Negative (for individual tags)

Settings are not namespaces, but are already of the format key:value.

A bit too lazy/too to provide instructions at the moment, but here's the aforementioned patch.

Hopefully it is soon outdated/not needed.

[horribleCodes]

After some googling, I managed to write a python script that exports the metadata of a png into a valid neighboring txt-file,
The tags can break a little with excessive use of prompt weighting and editing, but it does some formatting to keep your tags somewhat clean.
The negative prompt is imported as a single tag, since I figured you wouldn't really want to search for a negative prompt anyway. Ideally I'd like to import it as a note, but I don't think you can import that without custom scripting.

import os
from PIL import Image

def main():
    dirPath = "C:\\stable-diffusion-webui\\outputs\\txt2img-images"   # rename this to your folder of choice
    
    filelist = os.listdir(dirPath)

    for filename in reversed(filelist):
        if not(filename.endswith(".png")):
            continue

        if os.path.isfile(os.path.join(dirPath, filename+".txt")):
            continue

        im = Image.open(os.path.join(dirPath, filename))
        im.load()

        params = im.info['parameters'].lower()
        
        lines = params.split("\n")
        
        settings = getSettings(lines[len(lines)-1])
        
        lines.remove(lines[len(lines)-1])
        
        lineIsNegative = False
        
        tokens = []
        
        if len(lines) == 2:
            tokens = getTokens(lines[0])
            negatives = getNegatives(lines[1])
        else:
            for line in lines:
                strippedLine = line.strip()
                if strippedLine == "":
                    continue
                
                if lineIsNegative:
                    negatives += ", " + strippedLine
                    continue
                
                if strippedLine.startswith("negative prompt: "):
                    lineIsNegative = True
                    negatives = getNegatives(strippedLine)
                    continue
                
                tokens += getTokens(strippedLine)
                    
        writeNewFile(dirPath, filename, negatives, tokens, settings)

def getNegatives(line):
    negatives = line.replace("negative prompt: ","negative:",1)
    return negatives

def getSettings(line):
    setup = line.replace(": ",":")          # Removes the space between the namespace and tag
    settings = setup.split(",")
    settings = [setting.strip().replace(" ","_") + "\n" for setting in settings]
    return settings

def getTokens(line):
    prompt = line.replace(":",";")          # Replace : to avoid unwanted namespaces
    tokens = prompt.split(",")
    tokens = [token.strip().replace(" ","_") + "\n" for token in tokens]
    tokens = list(filter(None, tokens))
    return tokens

def writeNewFile(dirPath, filename, negatives, tokens, settings):
    sidecarFile = open(os.path.join(dirPath, filename+".txt"), "w", encoding="utf-8")
    sidecarFile.writelines(tokens)
    sidecarFile.writelines(settings)
    sidecarFile.write(negatives)
    sidecarFile.close()

if __name__ == '__main__':
     main()

To use it, replace the string in dirPath with the path of the image folder (no recursive search - sorry, directory users) and execute it. It will only create files for PNGs that don't have a corresponding text yet, and will skip everything else.

Edit: I added multiline support. Personally, I decided to not include negatives in the sidecar because I don't think it adds anything in organization, but I left it for completion's sake. If you also don't want them, just comment out the line sidecarFile.write(negatives).

[evanjs]

[Update for 516]

Dev has added the ability to import and export txt/json sidecars via the edit metadata migration routers dialog.
This will allow users to easily share configurations that include tag parsing rules for sidecar files, which can then be associated with import folders, allowing for fully automated importing and automatic tagging of stable diffusion generations if sidecar txt files are found.

[Update for 505]

Dev has added some functionality for EXIF parsing.

Going forward, the client will scan for the presence of embedded metadata in images, and will flag them as such.
Once you update to 505, jobs will be added to scan existing files.
You can either let them run "naturally" over the next few weeks, or make them run now, e.g.:

• database (menu item)
  • file maintenance
    • manage scheduled jobs
      • do all work

Currently, you can only view the embedded metadata from the Hydrus media viewer, but as indicated in the changelog for 505:

a future expansion here will be to record the specific keys and values into the database so you can search specifically over those values (e.g. 'EXIF ISO level > 400', or 'has "parameters" text value')

Considering this and the previously added sidecar expansion, please check out 505/505a+!

---

Below is OUTDATED: USE 505/505a+

---

For those building Hydrus from source, I have thrown something together for sidecar and metadata parsing.

This works with outputs (with embedded generation parameters) from both SD WebUI and NovelAI.

For metadata, prompts are now added to the notes Negative prompt and Prompt rather than tags, which have a smaller upper limit, which is easier to exceed with full prompts.

TODO:

• Prompts -> tags notes for sidecar txt import
• Handle additional data from NAI outputs (if easily accessible)

---

Update for 504: metadata parsing included, sidecar SD parameter auto-parsing omitted to test new sidecar import rework from upstream, but txt files from Auto's WebUI can still be detected as sidecar files.

[ghostsquad]

I was just looking at HydrusNetwork to solve the problem of storing and finding images as they are generated by Stable Diffusion, of which I generate hundreds a day.

I'm very interested in some integration here.

[ghostsquad]

@evanjs are you planning on opening a pull request upstream with your changes?

[evanjs]

Development processes for Hydrus aside, I do not plan on merging my changes, no.
They are quite outdated at this point, anyway.

To elaborate: anything below Below is OUTDATED: USE 505/505a+ was made before the developer added scanning/persistence for generation metadata via EXIF, as well as the expanded sidecar support to allow parsing of txt files generated by stable-diffusion-webui

---

If you update to the latest Hydrus, it will ensure all media is scanned for the presence of EXIF data.
This data can then be viewed on relevant files in Hydrus after it is processed.

On the Hydrus side, there are a few TODOs that dev has mentioned already:

• Make sidecar configurations exportable/sharable
• Search functionality for things like notes, and I think metadata (e.g. generation data stored in EXIF), etc.

I am not sure if configuring metadata/EXIF rules like how sidecar files can be configured and then associated with import folders is planned at this point.

---

The current features, however, allow for e.g. users to add import folders that can import sidecars from SD, using the new remove file .ext? toggle, like so

Users can then setup processing steps for the strings from the sidecar files using the button at the bottom with the pictured value of no string processing

---

I'd recommend users that want to utilize features relating to EXIF data and such update for the time being, if only so their media is scanned/indexed and ready to go once additional features surrounding metadata are released.

[space-nuko]

I made a better version of @horribleCodes' script that correctly parses the prompt attention and handles the prompt editing syntax. I also added compatibility with https://github.com/kohya-ss/sd-webui-additional-networks so LoRA model names/hashes are added as tags. It's available here:

https://github.com/space-nuko/sd-webui-utilities/blob/master/import_to_hydrus.py

It requires some extra dependencies so check the repo. I like using it as follows

python import_to_hydrus.py path/to/outputs/txt2img-images -tsite:personal -tgen_type:txt2img-images