Does inference with a PEFT model require all merged LoRA models or just the final saved LoRA and the base model?

[BGFGB]

Hi,
Let's say I have several LoRA models for Llama3. First, I merged them using the merge_and_unload function, and then I obtained a new LoRA model using get_peft_model from the merged model. If I train the new PEFT model and save a checkpoint, does the inference require all the LoRA models I merged, the final saved LoRA model, and Llama3? Or does it just need the Llama3 model and the final saved LoRA model? Thank you guys.

[BenjaminBossan]

Okay, so to make things clear: You have a base model B and you train a LoRA adapter L1. Then you merge them together, resulting in B+L1. Then you add a second LoRA adapter, L2, resulting in B+L1+L2, and train it. Now, when you call save_pretrained on the PEFT model, only L2 will be saved. Therefore, when you want to load it, you have to ensure that L1 is loaded too. The steps are thus:

1. load B
2. load L1 onto B
3. merge_and_unload to get B+L1 (same as you did before training L2)
4. load L2, resulting in B+L1+L2
5. optional: call merge_and_unload again to merge L2 into B+L1.

If for some reason, you do not want to go through all these steps to obtain B+L1+L2, what you can do is after step 5, call save_pretrained again. This will save the whole B+L1+L2 model, which you can just load with transformers (AutoModelForXxx.from_pretrained(...)). However, note that this checkpoint will be quite large.