SD-Hub

An extension for Stable Diffusion WebUI, designed to streamline your collection.
It lets you download files from sites like Civitai, Hugging Face, GitHub, and Google Drive, whether individually or in batch.
You can also upload files or entire folders to the Hugging Face model repository (with a WRITE token, of course), making sharing and access easier.
The extension also has functionality to archive and extract files in formats like tar.lz4, tar.gz, and zip.

Downloading/Uploading files from/to outside Models or Embeddings folders is blocked.
Add --enable-insecure-extension-access command line argument to proceed at your own risk.

Support both Windows and Unix.

Changelog

2024-12-25 v5.5.5

• Added security measures to restrict downloading, uploading, and compressing to the Models and Embeddings folders only when WebUI is run without --enable-insecure-extension-access.
  
• Added a click event listener to the Gradio DataFrame in Tag List to automatically copy the table text, making it easier to copy especially for mobile users.
  
• Added a function to automatically save the last used username, repository, and branch when uploading to Huggingface, which will be automatically used when WebUI is loaded.
  
• Fixed path handling.
  
• Relocated the token file to the extension folder and renamed it to .sd-hub-token.json.
  

2024-09-30 v4.8.4

• Gradio 4 Compatibility Update for Forge Webui.
  

2024-07-21 v4.5.6

• Added support for downloading from Civitai using webpage URLs directly.
  

2024-07-12 v4.4.4

• Added support for Windows.

2024-07-03 v3.3.3

• Added venv support.

2024-05-14 v3.2.1

• Added an optional argument - for the Scrape button to filter specific extension instead of using the default extension list.

https://huggingface.co/ckpt/controlnet-sdxl-1.0/tree/main - pth md txt safetensors

• Added an optional argument -- for the Uploader input box to exclude specific file extension instead of uploading all files.

$ext/sd-hub -- json txt py

• Moved Token.json to the Stable Diffusion root directory and renamed to sd-hub-token.json.
• Added a Split by radio button for the Archiver to split compressed files based on the total number of files if input is pointing to a folder.

2024-04-22 v2.0.2

• Added Scrape Button to return a list of Resolve URL from Huggingface repository, and Pastebin.
• Improved Compress and Decompress logic for Archiver.

Usage

Downloader

● Input box

Similar to batchlink, you use tag then URL:

$tag
URL

Tag should begin with $
Tag is mandatory and there is no default path.
For available tags, refer to the [Tag List] at the bottom of the extension.

Click on the table row, tag or path to automatically copy its text.

You can also add subdirectories to the tag if you have any:

$ckpt/tmp_ckpt
https://civitai.com/api/download/models/403131

To add an optional path:

$ckpt
https://civitai.com/api/download/models/403131 /kaggle/working/stable-diffusion-webui/zzzzz

To download with custom filename, add - after the URL or optional path (if provided):

# Without optional path
$ckpt
https://civitai.com/api/download/models/403131 - imagine-anime-XL.safetensors

# With optional path
$ckpt
https://civitai.com/api/download/models/403131 /kaggle/working/stable-diffusion-webui/zzzzz - imagine-anime-XL.safetensors

● Token box

Enter your Huggingface token with the role READ to download from your private repo, get one Here.
Enter your Civitai API key if you encounter an Authorization failed error. Get your key Here.
Save = To automatically load token upon Reload UI or Webui launch.
Load = Load token.

● Scrape Button

For Huggingface repository:
By default it will return a list of resolve URLs that match these extensions .safetensors .bin .pth .pt .ckpt .yaml
add - to return only a specific file extension.

Paste the repository URL in the following format:

# To scrape everything in the branch tree list (folders and subfolders won't be included)
htttps://huggingface.co/user_name/repo_name/tree/branch_name

# To filter specific extension
htttps://huggingface.co/user_name/repo_name/tree/branch_name - pth safetensors

# To Scrape a folder
htttps://huggingface.co/user_name/repo_name/tree/branch_name/folder

# or
htttps://huggingface.co/user_name/repo_name/tree/branch_name/folder/sub_folder

Enter your Hugginface READ token into Token box if you want to Scrape your private repo.

And Pastebin:
Simply paste the pastebin URL

https://pastebin.com/696969

And it will return a list of whatever is available at the pastebin URL.
If it has a hashtag from batchlink, it will automatically be replaced with SD-Hub Tags.

● Insert TXT Button

To upload a TXT file from your device, simply select it and upload it into the input box.

Supported Domains for Downloader: Civitai Huggingface Github Drive.Google

Uploader

● Input

Username = Your username at huggingface.co.
Repository = Your model repository at huggingface.co, it will automatically create a new repository if reponame does not exist.
Branch = Defaults to main. You can change the branch name to create a new branch.
Visibility = Defaults to Private and will only take effect if you are creating a new repository; otherwise, it will be ignored.
Token = Obtain your huggingface token with the role WRITE from Here.

For the input box, you can either provide a path pointing to a folder or a single file.
You can also use $tag to skip the long path name.
You can rename the upload (file or folder) by adding - after the input path.
You can exclude specific file extension from being uploaded by adding --.

# Folder as input, so all the files inside the folder with its folder uploaded to your repository
/kaggle/working/stable-diffusion-webui/models/Stable-diffusion

# With tag
$ckpt

# To rename the folder
$ckpt - my-merge

# To rename and exclude specific file extension
$ext/sd-hub - mymodel -- json txt py

# File as input, so only the file gets uploaded to your repository
/kaggle/working/stable-diffusion-webui/models/Stable-diffusion/animagineXLV31_v31.safetensors

# with tag
$ckpt/animagineXLV31_v31.safetensors

# to rename the uploaded file
$ckpt/animagineXLV31_v31.safetensors - XL-imagine-animeV31.txt

Archiver

Supported Format: tar.lz4 tar.gz zip

Archive:
Name Name for the compressed file (excluding the file extension).
Input Path Path pointing a single file or folder containing multiple files.
Output Path Path where the compressed file will be saved.
Create Directory Automatically creates a new folder at the Output Path if not already existing.
Split by Divide the compression into multiple files based on number of files in Input Path.

Extract:
Input Path Path pointing to a compressed file.
Output Path Path where the compressed file will be saved.
Create Directory Automatically creates a new folder at the Output Path if not already existing.

You can use $tag for the path in Input and Output Path.

# if input as a file, to compress a single file
/kaggle/working/stable-diffusion-webui/models/Stable-diffusion/animagineXLV31_v31.safetensors

# else input as a folder, to compress the whole files inside the input folder
/kaggle/working/stable-diffusion-webui/models/Stable-diffusion

# with Tag if input as a file
$ckpt/animagineXLV31_v31.safetensors

# with Tag if input as a folder
$ckpt

Credits

camenduru Thanks for the extension
etherealxx Thanks for the inspiration
Thanks to my Discord friends DEX-1101, VeonN4, kokomif, for always being there in the middle of the night.
Especially to cupang-afk, who helped me a lot with Python, thank you.