ICML

DoRA: Weight-Decomposed Low-Rank Adaptation
DoRA: Weight-Decomposed Low-Rank Adaptation
PEFT technique for efficient LLM/VLM adaptation. Significantly better than LoRA, supported in HF.
Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Min-Hung Chen
PDF Cite Code Project Video ICML2024(Oral)