PEFT means Parameter-Efficient Fine-Tuning.
It is the umbrella term for methods that update only a small slice of the weights.
LoRA, prefix tuning and adapters all sit under it.
Full fine-tuning needs many times the memory of the model itself.
PEFT makes customisation affordable and lets many variants share one base model.
PEFT stands for parameter-efficient fine-tuning, an umbrella rather than a single method.
The shared idea is to update a small slice of parameters and freeze the rest.
Full fine-tuning needs several times the model size in memory.
It also produces a complete copy of the model per task, which is expensive to store and serve.
Ten fine-tunes of a 7B model is 140 GB of near-identical weights.
PEFT replaces those copies with small task-specific pieces over one shared base.
LoRA is the dominant method and adds low-rank matrices beside existing weights.
Adapters insert small trainable modules between frozen layers.
Prefix and prompt tuning learn vectors prepended to the input instead of touching the model.
BitFit goes to the extreme and trains only the bias terms.
Quality is usually within a point or two of full fine-tuning on narrow tasks.
Full fine-tuning still wins when the target task differs greatly from pretraining.
PEFT also resists catastrophic forgetting, since the original weights are untouched.
Serving many adapters over one base is what makes per-customer models economic.
The Hugging Face PEFT library implements most of these behind one interface.
One question on this card. Every option explains itself once you pick, right or wrong.
15 concepts in this module, then a graded test with every answer explained.