About DyLoRA

[Zhouyuankun]

Hi there ! I just want to use DyLoRA from this paper Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low Rank Adaptation.. I could not found this usage in PEFT package. Is there any plan to add DyLoRA in the future ?

[BenjaminBossan]

No, this method is not implemented in PEFT. We mostly depend on outside contributors (often the paper authors) to contribute new methods for PEFT and DyLoRA was not contributed.

Judging from the abstract, it looks like it is similar to AdaLoRA, which is integrated in PEFT, so maybe you could give that a try.

[Zhouyuankun]

Thank you for this information !