frozen modules also be lora

[onehaitao]

System Info

• peft==0.7.1，transformers==4.45.0

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

model = get_peft_model(model, lora_config)

Expected behavior

hello, i use lora finetune for my model. I found that lora will enabled in freezed module. For example，my model constists of module A and module B, A is freezed. I found lora will be added into module A after model = get_peft_model(model, lora_config) called.

[BenjaminBossan]

In general, you can control which module is targeted with LoRA by defining the target_modules argument. The logic depends completely on the name of the module, PEFT does not check if a parameter is frozen or not.

[d-kleine]

Maybe it would be helpful to raise a warning message like "Parameter {name} was expected to be frozen but is now trainable."?

[BenjaminBossan]

Thanks for the suggestion, but I'm skeptical. This could lead to a lot of unnecessary warnings for users, when there was no warning previously. Having a lot of unnecessary warnings is dangerous as they can drown out important warnings and train the user to ignore them.

Moreover, the logic would not be easy to implement:

1. What if a module has both frozen and unfrozen parameters?
2. When adding a 2nd LoRA adapter, all base weights will be frozen after adding the 1st LoRA adapter, but we don't want to see warnings for those.

I can see that as a user, I could make the wrong assumption that @onehaitao did. But nowhere in PEFT do we suggest that the logic for adding LoRA layers is related to whether the base parameter is frozen or not. We specifically describe how to change the modules to target. Therefore, I'd say the warning is not worth the tradeoff.

The only thing I could see us adding is a mention in an appropriate place in the docs that the logic for modules to be targeted is not depending on whether the base weight is frozen.

[d-kleine]

It would only check if frozen params will be unfrozen (due to peft). But yeah, the second point might be a hurden. Adding this info to the docs is, I think, the best way.

[BenjaminBossan]

It would only check if frozen params will be unfrozen (due to peft)

Note that this should not happen, the base model parameters stay frozen. If I understood @onehaitao correctly, their complaint is that LoRA is applied to the frozen param, not that this param is unfrozen.

[d-kleine]

their complaint is that LoRA is applied to the frozen param, not that this param is unfrozen.

Ah, you're right - sorry, I misunderstood the question ("modules" confused me here). Yeah, but I mean, that's the principle idea of LoRA 🤷🏻‍♂️

[onehaitao]

Maybe it would be helpful to raise a warning message like "Parameter {name} was expected to be frozen but is now trainable."?

I think it will be better

[onehaitao]

Well, I mean that if a frozen module is added to lora，original logic is changed (frozen module not participating in training)

[BenjaminBossan]

I understand that this could come as a surprise to some, but for the reasons given above, we don't want to make the LoRA targeting dependent on whether a module is frozen and I explained why giving warnings for this problematic.

It is a good and common practice to print the model after applying PEFT to double check that it was applied to the desired modules. You can also inspect model.targeted_module_names to see which layers have been targeted.

[onehaitao]

okay, I am currently using this method, but when the model is relatively complex, it seems not intelligent enough and can easily make mistakes

[BenjaminBossan]

If you have trouble targeting (or not targeting) specific modules, let me know. The config options should be quite flexible and handle almost all use cases.

[d-kleine]

You might also check out the troubleshooting page, for example

• peft_model.print_trainable_parameters() for the num of trainable params
• peft_model.get_layer_status() for an info for each layer
• peft_model.get_model_status() for an info about the adapter layer status

etc.

[onehaitao]

Thanks for your help very much. I have solved my problem by targeting specific modules. If you have any suggestions, please reply to me.

from peft.tuners.tuners_utils import check_target_module_exists
freeze_params = [name for name, param in model.named_parameters() if not param.requires_grad]
trainable_target_modules = []
for module_name, _ in model.named_modules():
    if not check_target_module_exists(lora_config, module_name):
        continue
    if not any(param_name.startswith(module_name) for param_name in freeze_params):
        trainable_target_modules.append(module_name)
lora_config.target_modules = trainable_target_modules

[onehaitao]

My model consists of a LLM and a ViT, ViT is frozen and LLM is unfrozen. ViT and LLM both have similar transformer layers, so if I target modules, it will easily take effect in both ViT and LLM if target modules is linear_q/linear_k/linear_v. What I can think of is the method mentioned above. If peft supports this function or similar abilities , I think it will be more friendly.

[BenjaminBossan]

Glad you found a way that works for you. Just in case you did not know, target_modules can also be defined as a str, in which case it is treated as a regex, which might help you distinguish between the vision and the language part of your model.