Comfortable LORAs

[Tumanov87]

Hi! Just compared your Framepack version with colinurbs'es FramePack-Studio. And must say, your's works much better with Hunyuan video LORAS than his. But can you please do LORA adding process more comfortable? It take a lot of time to write paths to LORAs every time.
Thanks for your work!

[DrBlou]

Hello, same here :) and I juste cant figure how to add multiple loras separated by comma tho. Can you show an exemple please ? Ofc it would be better to have a dropdown list in the ui :) Thanks a lot for your work :)

[neph1]

My PR was merely done as a proof-of-concept. I don't really have an intention to make an easy to use app out of it. So, sorry if I won't make it easier to load loras :) Let's wait and see what happens with Framepack-F1. My hope is that these PR's will diffuse down into other tools.

@DrBlou Are you adding a space between the comma and the next lora? It's a common mistake. Just type:
path_to_lora1,path_to_lora2
There was a discussion here: #11
But it seems it's hidden for some reason.

Edit: Still,though, if I do update it; What would you consider more comfortable? Would a 'lora' folder be enough? It would mean a shorter path. Or a file selector?

[Tumanov87]

My PR was merely done as a proof-of-concept. I don't really have an intention to make an easy to use app out of it. So, sorry if I won't make it easier to load loras :) Let's wait and see what happens with Framepack-F1. My hope is that these PR's will diffuse down into other tools.

@DrBlou Are you adding a space between the comma and the next lora? It's a common mistake. Just type: path_to_lora1,path_to_lora2 There was a discussion here: #11 But it seems it's hidden for some reason.

Edit: Still,though, if I do update it; What would you consider more comfortable? Would a 'lora' folder be enough? It would mean a shorter path. Or a file selector?

'lora' folder is enough, as for me! But file selector reduces a lot of issues.

[kacecarol]

There is no clear instruction about this modification, I can only see "To use lora, launch with the '--lora path_to_lora' argument. Can pass several comma-separated loras. Strength configurable with slider"

but I'm noob and no clue about this,
1.where to "add --lora path_to_lora"
2.where to put my safetensors file
3.where is the model_config.json file
4.and how to set the strength

I hope there can be an example .py/.bat/.json file to illustrate, a sample operation screen record clip to youtube/bilibili will be better.

[Denarius40]

There is no clear instruction about this modification, I can only see "To use lora, launch with the '--lora path_to_lora' argument. Can pass several comma-separated loras. Strength configurable with slider"

but I'm noob and no clue about this, 1.where to "add --lora path_to_lora" 2.where to put my safetensors file 3.where is the model_config.json file 4.and how to set the strength

I hope there can be an example .py/.bat/.json file to illustrate, a sample operation screen record clip to youtube/bilibili will be better.

It's always worth it to ask ChatGPT or another LLM for help with this. Here is what ChatGPT said based on your questions and a link to this repo:

1. Adding the --lora Argument
   To use a LoRA model, you'll need to launch the application with the --lora argument, specifying the path to your .safetensors file. If you're using the provided demo_gradio.py script, you can modify it as follows:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--lora', type=str, help='Path to LoRA .safetensors file')
args = parser.parse_args()

# Later in your code, use args.lora to load the LoRA model

Alternatively, if you're launching the script from the command line, you can pass the argument directly:

python demo_gradio.py --lora path/to/your/lora_model.safetensors

You can also specify multiple LoRA models by separating their paths with commas:

python demo_gradio.py --lora path/to/lora1.safetensors,path/to/lora2.safetensors

2. Placing Your .safetensors File
   Place your .safetensors LoRA files in a directory that's accessible to your script. A common practice is to create a loras directory within your project:

FramePack/
├── demo_gradio.py
├── loras/
│   ├── lora_model1.safetensors
│   └── lora_model2.safetensors

Then, when launching the script, reference the path accordingly:

python demo_gradio.py --lora loras/lora_model1.safetensors

3. Handling model_config.json
   According to the repository's README, the model_config.json has been replaced to simplify usage. Therefore, you don't need to worry about this file when integrating LoRA models.

4. Setting the Strength
   The strength of the LoRA model can be adjusted via a slider in the Gradio interface. Once you run demo_gradio.py, a web interface will launch, typically at http://localhost:7860. In this interface, look for a slider labeled "LoRA Strength" or similar, which allows you to adjust the influence of the LoRA model on the output.

5. Example .bat File for Windows Users
   If you're on Windows and prefer using a .bat file to launch the application with your LoRA model, here's an example:

@echo off
python demo_gradio.py --lora loras\lora_model1.safetensors
pause

Save this as run_with_lora.bat in your FramePack directory and double-click to execute.

[Tumanov87]

There is no clear instruction about this modification, I can only see "To use lora, launch with the '--lora path_to_lora' argument. Can pass several comma-separated loras. Strength configurable with slider"
but I'm noob and no clue about this, 1.where to "add --lora path_to_lora" 2.where to put my safetensors file 3.where is the model_config.json file 4.and how to set the strength
I hope there can be an example .py/.bat/.json file to illustrate, a sample operation screen record clip to youtube/bilibili will be better.

It's always worth it to ask ChatGPT or another LLM for help with this. Here is what ChatGPT said based on your questions and a link to this repo:

1. Adding the --lora Argument
   To use a LoRA model, you'll need to launch the application with the --lora argument, specifying the path to your .safetensors file. If you're using the provided demo_gradio.py script, you can modify it as follows:

import argparse

parser = argparse.ArgumentParser()
parser.add_argument('--lora', type=str, help='Path to LoRA .safetensors file')
args = parser.parse_args()

# Later in your code, use args.lora to load the LoRA model

Alternatively, if you're launching the script from the command line, you can pass the argument directly:

python demo_gradio.py --lora path/to/your/lora_model.safetensors

You can also specify multiple LoRA models by separating their paths with commas:

python demo_gradio.py --lora path/to/lora1.safetensors,path/to/lora2.safetensors

2. Placing Your .safetensors File
   Place your .safetensors LoRA files in a directory that's accessible to your script. A common practice is to create a loras directory within your project:

FramePack/
├── demo_gradio.py
├── loras/
│   ├── lora_model1.safetensors
│   └── lora_model2.safetensors

Then, when launching the script, reference the path accordingly:

python demo_gradio.py --lora loras/lora_model1.safetensors

3. Handling model_config.json
   According to the repository's README, the model_config.json has been replaced to simplify usage. Therefore, you don't need to worry about this file when integrating LoRA models.
4. Setting the Strength
   The strength of the LoRA model can be adjusted via a slider in the Gradio interface. Once you run demo_gradio.py, a web interface will launch, typically at http://localhost:7860. In this interface, look for a slider labeled "LoRA Strength" or similar, which allows you to adjust the influence of the LoRA model on the output.
5. Example .bat File for Windows Users
   If you're on Windows and prefer using a .bat file to launch the application with your LoRA model, here's an example:

@echo off
python demo_gradio.py --lora loras\lora_model1.safetensors
pause

Save this as run_with_lora.bat in your FramePack directory and double-click to execute.

HERE's the example of run.bat string:

"%DIR%\python\python.exe" demo_gradio.py --server 127.0.0.1 --inbrowser --lora "F:\FRAMEPACK\LORA_1.safetensors","F:\FRAMEPACK\LORA_2.safetensors"

where "F:\FRAMEPACK" - is the path to LORA on your PC, "LORA_1.safetensors" -is the name of LORA

[yucls]

Later in your code, use args.lora to load the LoRA model
Can you tell me specifically how to modify this step? I have tried several methods but have not been successful. This is my code
if args.lora:
transformer = HunyuanVideoTransformer3DModelPacked.from_pretrained(
'lllyasviel/FramePackI2V_HY',
torch_dtype=torch.bfloat16,
local_files_only=False,
)
transformer.load_attn_procs(args.lora)
else:
transformer = HunyuanVideoTransformer3DModelPacked.from_pretrained('lllyasviel/FramePackI2V_HY', torch_dtype=torch.bfloat16).cpu()