Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IMPROVE LORA DATASET LABELING] - Trim labels #16121

Closed
wants to merge 187 commits into from
Closed

[IMPROVE LORA DATASET LABELING] - Trim labels #16121

wants to merge 187 commits into from

Conversation

snghong
Copy link

@snghong snghong commented Jun 30, 2024

As per Improve Lora Dataset Labeling, we remove global identifiers from local lora train dataset labels to reduce noise:

  • trim local dataset labels by removing main prompt tags

Evan and others added 30 commits March 19, 2024 21:16
changes
f8bf163
chmod
f8391b8
f
dd08b9b
ignore
f12f962
files
b2d2924
config
069ab3e
f
872988e
@evanhu1
Update README.md
02d077a
@evanhu1
Update README.md
ec97a92
readme
8eca4cf
config
5b8c216
readme
8cd0d53
v
cc459d1
@w-e-w
fix initial corrupt model loop
2b717bb 
if for some reason the initial loading model at loading phase of webui  is corrupted
after entering this state the user will not be able to load even a good model is selected, due the the unload_model_weights  > send_model_to_cpu > m.lowvram attribute check will fail becaules m is None
webui will be stuck in the loop unable to recover without manual intervention
@w-e-w
Show LoRA if model is None
4bc39d2
@w-e-w
add option to check file hash after download
246c269 
if the sha256 hash does not match it will be automatically deleted
@w-e-w
ensure integrity for initial sd model download
c69773d
@drhead
add code for skipping CFG on early steps
a1aa0af
@drhead
add option for early cfg skip
8016d78
@drhead
Add KL Optimal scheduler
8326620
@drhead
change skip early cond option name and to float
83182d2
@drhead
change skip_early_cond code to use float
6e9b69a
@drhead
add s_min_uncond_all option
33cbbf9
@drhead
implement option to skip uncond on all steps below ngms
029adbe
@pinanew
AVIF has quality setting too
50bb6e1
@w-e-w
fix exif_bytes referenced before assignment
8fa3fa7
@w-e-w
update jpeg_quality description
1091e3a
@light-and-ray
more extension tag filtering options
e85e327
@w-e-w
compact-checkbox-group
d5f6fdb
@brendanhoar
Better error handling to skip non-standard ss_tag_frequency content
3902aa2
bluelovers and others added 27 commits June 9, 2024 19:07
@bluelovers
feat: save pattern add basename
6447ff4 
`grid` or `xyz_grid` or `img`

```py
'basename': lambda self: 'img' if self.basename == '' else self.basename,
```
@AUTOMATIC1111
performance: check for nans in unet only once, after all steps have b…
6214aa7 
…een completed
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15978 from bluelovers/pr/pattern-001
981abbb 
feat: save pattern add `basename`
@AUTOMATIC1111
stylistic changes for AUTOMATIC1111#15978
e368cd2
@AUTOMATIC1111
lint
aafbb5b
@AUTOMATIC1111
added tool for profiling code
57e6d05
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15823 from drhead/patch-3
1d0bb39 
[Performance] Keep sigmas on CPU
@AUTOMATIC1111
undo some changes from AUTOMATIC1111#15823 and fix whitespace
99e65ec
@AUTOMATIC1111
added onOptionsAvailable callback for javascript for
d2097db
@w-e-w
.gitignore trace.json
74ee8fd
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15980 from AUTOMATIC1111/git-ignore-…
a84000c 
…trace.json

.gitignore trace.json
@huchenlei
Add process_before_every_sampling hook
17e8461
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15984 from huchenlei/before_every_sa…
e33bb8f 
…mpling

Add process_before_every_sampling hook
@w-e-w
multi size grid
abacb73
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15988 from AUTOMATIC1111/multi-size-…
123582b 
…grid

multi size grid
@silveroxides
Add option to enable clip skip for clip L on SDXL
00e0938
@silveroxides
Update sd_hijack_clip.py
91ecc75
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#15992 from silveroxides/dev
d5e2627 
Add option to enable clip skip for clip L on SDXL
@zero41120
feat: prevent screen sleep during generation
1f8f3a6
@zero41120
fix: prevent create multiple wake lock
c803e11
@zero41120
ci: remove comments and console logs
f1e0bfe
@AUTOMATIC1111
Merge pull request AUTOMATIC1111#16001 from zero41120/feat-prevent-sc…
a30b19d 
…reen-lock

feat: Prevent screen sleep during generation
merge
e60a8c4
stable
b35249a
stable
87794a2
@snghong
[IMPROVE LORA DATASET LABELING] trim dataset labels by removing main …
fac49c1 
…prompt words
@snghong
[IMPROVE LORA DATASET LABELING] trim dataset labels by removing main …
2c45679 
…prompt words
@snghong snghong requested a review from AUTOMATIC1111 as a code owner June 30, 2024 17:08
@snghong snghong changed the title Trim labels [IMPROVE LORA DATASET LABELING] - Trim labels Jun 30, 2024
@w-e-w w-e-w closed this Jul 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AUTOMATIC1111 AUTOMATIC1111 Awaiting requested review from AUTOMATIC1111 AUTOMATIC1111 is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.