Training & Learning

Low-Rank Adaptation

LoRA

An efficient fine-tuning method that freezes the original model and trains only a small set of added weights.

LoRA makes fine-tuning cheap by leaving the huge pretrained model frozen and learning only a small pair of low-rank matrices that nudge its behavior. Because you train a tiny fraction of the parameters, it slashes the memory and compute a full fine-tune would need, often by more than half, with little loss in quality. It is the most popular form of parameter-efficient fine-tuning, and the small adapters are easy to store and swap per task.