1) Filter common conversational words and characters, 2) make output imgs cheap by linking, 3) get models from SD-server and let user choose #3
Conversation
1) Added the function filter_out_conversational_words, which filters out common english conversational words that ~confuse SD models. 2) Added call to filter_out_conversational_words in the get_SD_pictures function, so that the response from the chatbot is filtered when passed to AUTOMATIC1111's SD api, but still presented unfiltered to the user of oobabooga. I am submitting this to the last published version of the script on a repo that seems outdated compared to oobabooga's. However, in oobabooga's repo this is mentioned as the experimental branch so, so to not mess things up, the chages here (originally made to the updated script in the obabooga repo) are incorporated in the old script present in the 'Brawlence/SD_api_pics' repo.
Removed description of purpose of fork. Here is the purpose of the fork: 1) Added the function filter_out_conversational_words, which filters out common english conversational words that ~confuse SD models. 2) Added call to filter_out_conversational_words in the get_SD_pictures function, so that the response from the chatbot is filtered when passed to AUTOMATIC1111's SD api, but still presented unfiltered to the user of oobabooga. I am submitting this to the last published version of the script on a repo that seems outdated compared to oobabooga's. However, in oobabooga's repo this is mentioned as the experimental branch so, so to not mess things up, the chages here (originally made to the updated script in the obabooga repo) are incorporated in the old script present in the 'Brawlence/SD_api_pics' repo.
…en taken up by resulting images. This push fixes the problem of excessive tokens taken up by putting results of SD-generation results taking up too many token. The solution is very simple: Provide a link (it is a link to cached file in browser, not to one saved to harddrive) to image. I had to update the old 'not-updated-code' (in comparison to the one in oobabooga), but this enhancement works the same nonetheless. I recommend updating the code in this repo to at least match the one in oobabooga. Include my enhancement or don't but please update the code in the repo marked as the experimental one; or encourage updates directly in the main oobabooga repo.
|
Sorry about the typos in the descriptions :/ Just ask, if something in incomprehensible :) |
|
The idea has merit. First, I wanted to argue that repeated substring searches are not performant, but... I'm kinda surprised by the benchmark results. The output string is a mess of words broken by commas, sometimes the things left are outright non-descriptive. Second, I don't believe the header statement; after all, (natural descriptions - picture) pairs are what CLIP was trained on. Stable Diffusion checkpoints include CLIP, so they should perform well. Well, maybe if one uses NovelAI-based checkpoints -- then yes, it's less tolerant to natural language descriptions and responds better to danbooru-style tags instead. I can't decide yet if the PR is worth it. Check this out in the meantime: there was another PR in the main repo which tried to improve prompt tags in another way. As for the tokens, this fixes the problem that shouldn't even exist in the first place (if it even exists, cause I know for a fact that ooba logs two histories, one visible for the UI and the one hidden for the model itself) — non-text inputs should not be forwarded to the model at all. This is relevant not only to SD_api_pics, but also to TTS & similar extensions that use embeddings. |
Brawlence
added
enhancement
Regarding the jumbled collection of words (mess) resulting from the operation of filter_out_conversational_words.Yes, the output string is jumbled, but from a decent range of tests before implementing the filter and more than a 100 tests after implementing it, I can safely postulate that a lot of SD models like the jumbled strings better (give results closer to the intent of the unjumbled text; and the unjumbled text is still display to the user). Regarding the speed of the functionYes. The function is relatively slow. 0.74 secs on my pc (running your test code). That is pretty bad, but for this type of application I believe the 0.7 seconds (for a 1000 requests) is acceptable. However, by removing the second run of deletions of elements from substrings_to_remove-variable in the string-variable, the test time goes does to 0.43 secs. And the quality of the string resulting from the removal is not reduced significantly. The token thingYeah, it is really wierd. I tested a bit before deciding on the link-solution. I regret posting it now, because it is not satisfactory. We really want the images to appear on the same page. But it does 'solve' the problem in a way that is understandable to non-tech users. (And since most users have GPUs with less than 12/16GB of VRAM removing these tokens really is a significant optimization.) |
…meters Section Also converted previous enhancement suggestions to the newly updated version of the script.
kiancn
changed the title
kiancn
changed the title
Before it displayed the full list (which, admidtedly had only one member). It (initialization of the name displayed in dropdown) still does not work properly - initially no no string is shown. Reason not yet understood by me.
Commented with wrong method name corrected.
Remove unused function
* Refactored: removed sd_model_current variable, for the same purposes now using params['SD_model'] instead. * Added a row in the ui for the models dropdown.
Model selection now completely functional in the UIIt seemed like something that was missing.
|
|
The model selection is currently breaking VRAM management options. Gotta think what's up with that |
|
That is funky... However, for me, the VRAM management never worked. I've got responses like this one with every request the extension sends from the give_VRAM_priority function: I'll fix my install so I can at least experience the same error as the up-to-date user :) |
|
Ok. I updated the relevant parts of the code (in Automatic1111 codebase) to be up-to-date, and now VRAM-management works for me. Which puzzles me. I played around a bit and didn't get the extension to fail. @Brawlence Could you describe the error you experience? |
2 participants
I had to update the old 'not-updated-code' (in comparison to the one in oobabooga), but this enhancement works the same nonetheless.
I recommend updating the code in this repo to at least match the one in oobabooga. Include my enhancement or don't but please update the code in the repo marked as the experimental one; or encourage updates directly in the main oobabooga repo.