Training & Learning
Low-Rank Adaptation
LoRAAn efficient fine-tuning method that freezes the original model and trains only a small set of added weights.
LoRA makes fine-tuning cheap by leaving the huge pretrained model frozen and learning only a small pair of low-rank matrices that nudge its behavior. Because you train a tiny fraction of the parameters, it slashes the memory and compute a full fine-tune would need, often by more than half, with little loss in quality. It is the most popular form of parameter-efficient fine-tuning, and the small adapters are easy to store and swap per task.
Related terms
Fine-Tuning
Taking a pretrained model and training it further on a smaller, focused dataset to specialize it for a task or style.
Parameters
The adjustable numbers inside a model that get tuned during training and store what the model has learned.
Instruction Tuning
Fine-tuning a base model on examples of instructions and good responses so it follows requests instead of just continuing text.
Quantization
Shrinking a model by storing its numbers at lower precision, so it runs faster and on smaller hardware.