Some parameters of the SD3 is not updated in dreambooth lora fine-tuning

[ZYM-PKU]

In train_dreambooth_lora_sd3.py, the lora matrices are added to the "to_q, to_k, to_v, to_out.0" modules, which seem to be the projection modules of image embeddings, but the "add_q_proj, add_k_proj, add_v_proj, to_add_out" modules (projection modules of text embeddings) is not considered. Why is that? Does it mean the "add_" modules not important when finetuning sd3?

transformer_lora_config = LoraConfig( r=args.rank, lora_alpha=args.rank, init_lora_weights="gaussian", target_modules=["to_k", "to_q", "to_v", "to_out.0"], )