I made a fully AI powered prompt enhancer with GPT-Neo; Integrate? #1703
Replies: 3 comments 3 replies
-
|
You should only use the SD prompts from Lexica, since MJ uses all kinds of postprocess for their gens, also the Kreita guys github has the entire 10M SD prompts from the Discord beta. I am finishing a project, but I'll come back and can help you get a better data set for this. |
Beta Was this translation helpful? Give feedback.
-
|
The prompts I used are included if you want to reiview them. I was concerned if the optimal MJ promt was significantly different from the optimal SD prompt, and I eyeballed them a bit; but the same sorts of keywords seem to improve them both; and the MJ prompts in "favorites" were more strongly sorted for quality and very numerous, so I decided to use them. One issue is that the ones that were easily scrape-able on discord were also truncated, so they might shorten the average length of generated prompts, which annoys me. I found where to turn off attention from EOT symbol during training for truncated texts, so it won't actually generate end of text prematurely(I think); so maybe prompt length isn't affected; I'm not sure. I would love to rip the "top" from their community feed since those are REALLY high quality prompts, but I can't figure out how yet; it's javascript or something popping them up, not simple html. Lexica prompts (that come up at the front of the webpage anyway) seemed much better than average ones made a casual user, so I was happy to use those as well. Plus I'm more interested in a tool that will support SD than MJ. I will check out Kreita's page; would you say the prompts have some level of quality control? Might be useful regardless, but that would be really nice. One important thing to remember is that generating them with GPT-Neo is probably overkill - bigger models in the same family have convinced people they are sentient. So I could easily get it to generate MJ, SD, and "general" prompts seperately with almost no additional work, and it probably would still only be using a fraction of the thing's processing power. Literally just prefix "prompt: ", "MJ_prompt: ", or "SD_prompt: " in front of the training/testing sequences, repeat them all twice(an extra time as generic), and filter out the prefix from any outputs. I am thinking about whether I should bother complicating matters by doing that though. I would love any help getting datasets that you could give! Thnks for pointing me at Kreita. My model takes like 500MB, do you have any idea what the standard [non pay] protocol is for sharing big datafiles that go along with github projects? |
Beta Was this translation helpful? Give feedback.
-
|
I like the idea of making the prompt more powerful, also interactivity to enhance the prompt sounds like a good approach. Regarding Lexica: it's a cool project, helpful for sure but is it the way forward ? I don't think so. So Lexica does not help to decide (without looking and researching) if a prompt helped to get a certain feature and over time the prompts in lexica will become more and more outdated. |
Beta Was this translation helpful? Give feedback.
-
|
? There is not an online API connection. I just included the code I used to scrape the training set in case anyone wanted to use it to scrape more training data. I think you have a great point that knowledge about optimal prompts is still changing rapidly; that's exactly why it's good to bundle the dataset & scraper code, and to be looking for better datasets in future. The majority of good prompts just use a handful of tricks people have learned so far, I'm sure there's a LOT of room for improvement/diversity. One interesting idea would be letting an AI generate random prompts and then analyze the quality[via human or AI] to try to have it discover new rules. I would question your assertion that new model versions require differently formed prompts, at least so far; on what do you base this idea? Optimal prompt should be a function of the training set and labeling method, I think architecture only matters if the training method is actually significantly different. The training sets clearly vary between models; I'm less sure they vary much within a model at different versions. But I think that's beside the point. If one specific model doesn't know about Akira Toriyama, or what a crossbow is, I think it's very arguable whether it is desired behavior to avoid mentioning them in prompts. Knowledge of stuff like classical styles, art mediums, names of artists, industry terms for lighting and photography methodologies, etc. is contained in the model; and generating them for laymen, so they don't need to be an art expert is one of the main points of a model like this. And that knowledge is static, completely independent of model architecture and even model training. Anyway, I think as long as certain keywords appear more often near good art than bad art in our training sets("featured on artstation", the name of a skilled artist, "UHD") then the things that constitute a 'good' prompt are unlikely to change that much. The prompts on the front page of lexica seem better than "average" randomly generated prompts, the fact that they DO seem to describe the content quality[for whatever reason] is why I decided to use them. Of course, I would LOVE if there was a source of labeled data with a quality measure on it; the 2 sources I used were the closest I could find in bulk. I would say that lexica DOES help decide if a prompt helped get a certain feature over time (assuming I'm right that the featured images are above average in some way), because skilled users naturally will try to produce desirable features, including eyes, fingers, whatever. I think the 5 finger thing and general inability to handle multiple objects is a product of deep architectural choices, and yeah, prompts will be very different when we have architectures that can handle them. Almost everything that works now should still work imo; but there will be a lot more aspects of the images that will be tuneable, and prompts to do it. But yeah, we definitely need better (and up-to-date) datasets to make something like this work as well as possible. |
Beta Was this translation helpful? Give feedback.
-
|
Lexica is also only based on the Aug6-20 Discord beta from SD. It doesn't update over time. I am working on the dynamic content qualifier problem right now, almost done w V1. |
Beta Was this translation helpful? Give feedback.
-
|
I didn't know lexica doesn't update. Annoying. Dynamic content qualifier? |
Beta Was this translation helpful? Give feedback.
-
Still very crude, but it does seem to work:
https://github.com/DiBianoR/Prompt_Enhancer
Yall should integrate this into the project rather than just have it add 1-2 artist names like it does now. I'd love to contribute; what do I do to make that happen?
It's currently trained with GPT-Neo on prompts from the "favorites" channel of midjourney, and prompts from the top of lexica(which seem to be better than average quality). If anyone knows another good source of "quality" prompts, msg me.
Beta Was this translation helpful? Give feedback.
All reactions