LoRA config in finetune.py

[marcoyang1998]

Hi,

Thank you for the great work!

I noticed that the default LoRA config for finetuning is:

--lora_rank=64
--lora_alpha=2

The lora_alpha is very small. Is there a reason for that? Did you find this to be beneficial?

[Benioh]

Hi,

Thanks for pointing this out!

The LoRA config in the finetuning script is mainly provided as an example/default setting rather than a carefully tuned recommendation. We have not systematically tested different lora_alpha values, so there is no specific empirical reason behind choosing lora_alpha=2.