Code

flux support with fp8 freeze model#16484

Code

Open

wkpark wants to merge 50 commits intoAUTOMATIC1111:devAUTOMATIC1111/stable-diffusion-webui:devfrom wkpark:minimal-flux-with-fp8-freezewkpark/stable-diffusion-webui:minimal-flux-with-fp8-freezeCopy head branch name to clipboard

+1,689-127

Commits

Commits on Nov 1, 2024

• import Flux from https://github.com/black-forest-labs/flux/

  Show description for 853551b
  wkpark

  committed
  853551b

• add flux model wrapper

  wkpark

  committed
  d38732e

• fix for flux

  wkpark

  committed
  2d1db1a

• use empty_like for speed

  wkpark

  committed
  821e76a

• fix misc

  Show description for 39328bd
  wkpark

  committed
  39328bd

• check Unet/VAE and load as is

  Show description for c972951
  wkpark

  committed
  c972951

• simplified get_loadable_dtype

  wkpark

  committed
  fcd609f

• misc fixes to support float8 dtype_unet

  Show description for 537d9dd
  wkpark

  committed
  537d9dd

• add shared.opts.lora_without_backup_weight option to reduce ram usage

  wkpark

  committed
  2060886

• support copy option to reduce ram usage

  wkpark

  committed
  2f72fd8

• fix to support dtype_inference != dtype case

  wkpark

  committed
  24f2c1b

• preserve detected dtype_inference

  wkpark

  committed
  477ff35

• add diffusers weight mapping for flux lora

  Show description for d6a609a
  wkpark

  committed
  d6a609a

• fix for Lora flux

  wkpark

  committed
  51c2852

• fix for t5xxl

  wkpark

  committed
  7e2d519

• vae fix for flux

  wkpark

  committed
  9c0fd83

• support Flux1

  wkpark

  committed
  219a0e2

• fix to support float8_*

  wkpark

  committed
  9e57c72

• add cheap approximation for flux

  wkpark

  committed
  789bfc7

• minor update

  Show description for 44a8480
  wkpark

  committed
  44a8480

• pytest with --precision full --no-half

  wkpark

  committed
  9617f15

• fix lora without backup

  wkpark

  committed
  3cdc26a

• revert to use without_autocast()

  wkpark

  committed
  3b18b6f

• fix position_ids

  wkpark

  committed
  2ffdf01

• fix for float8_e5m2 freeze model

  wkpark

  committed
  1e73a28

• fix load_vae() to check size mismatch

  wkpark

  committed
  1318f61

• add fix_unet_prefix() to support unet only checkpoints

  wkpark

  committed
  eee7294

• use assign=True for some cases

  wkpark

  committed
  6675d1f

• check lora_unet prefix to support Black Forest Labs's lora

  wkpark

  committed
  1f77922

• call lowvram.send_everything_to_cpu() for interrupted case

  wkpark

  committed
  380e9a8

• call torch_gc() to fix VRAM usage spike when call decode_first_stage()

  wkpark

  committed
  71b430f

• use text_encoders.t5xxl.transformer.shared.weight tokens weights

  Show description for f569f6e
  wkpark

  committed
  f569f6e

• fix flux to use float8 t5xxl

  wkpark

  committed
  28eca46

• fixed typo in the flux lora map

  wkpark

  committed
  4bea93b

• fixed ai-toolkit flux lora support

  Show description for 30d0f95
  wkpark

  committed
  30d0f95

• make Sd3T5 shared.opts.sd3_enable_t5 independent

  wkpark

  committed
  11c9bc7

• support Flux schnell and cleanup

  wkpark

  committed
  8c9c139

• fix some nn.Embedding to set dtype=float32 for some float8 freeze model

  wkpark

  committed
  2e65335

• use isinstance()

  wkpark

  committed
  03516f4

• use shared.opts.lora_without_backup_weight option in the devices.autocast()

  Show description for ba499f9
  wkpark

  committed
  ba499f9

• do not use copy option for nn.Embedding

  wkpark

  committed
  5f3314e

• do not use assing=True for nn.LayerNorm

  wkpark

  committed
  4ad5f22

• flux: clean up some dead code

  wkpark

  committed
  98cb284

• task manager added

  Show description for 1d3dae1
  wkpark

  committed
  1d3dae1

• reduce backup_weight size for float8 freeze model

  wkpark

  committed
  0ab4d79

• extract backup/restore io-bound operations out of forward hooks to speed up

  wkpark

  committed
  04f9084

• call gc.collect() when wanted_names == ()

  wkpark

  committed
  2a1988f

• backup only for needed weights required by lora

  wkpark

  committed
  412401b

• fix for lazy backup

  wkpark

  committed
  b783a96

• restore org_dtype != compute dtype case

  wkpark

  committed
  310d0e6