Upvotes
0
Pricing
Open Source
Category
Coding Tools
In the database since
Oct 2026
Overview
About PEFT
PEFT is Hugging Face's open-source library for parameter-efficient fine-tuning of large models. Fine-tuning a whole model updates billions of weights, which costs a lot of memory and storage. PEFT instead trains only a small set of extra parameters, called adapters, while the base model stays frozen. That makes customizing big models for your own tasks cheap and practical, even on modest hardware. It works with Transformers, Diffusers, and other Hugging Face libraries, and it fits into existing training code with small changes.
Key Features
LoRA
LoRA trains low-rank adapter matrices instead of full weights. You get full-model quality at a fraction of the compute and memory.
QLoRA
QLoRA combines LoRA with 4-bit quantization for even bigger savings. Huge models fit on modest GPUs.
Many Methods
PEFT includes adapters, prefix tuning, and more techniques. One API covers them all. Try different methods without rewriting your code.
Hugging Face Native
It plugs into Transformers, Diffusers, and TRL seamlessly. Your existing code barely changes. Adapters save as small, shareable files.
Mergeable Adapters
Trained adapters can merge back into the base model. Deployment stays simple with one file.
Frequently Asked Questions (FAQ)
How do I install PEFT?
Install it with pip install peft, then wrap your model with get_peft_model and a config like LoraConfig. The quickstart shows a full working example.
What is LoRA?
LoRA (Low-Rank Adaptation) trains tiny weight matrices that plug into a frozen model. It gives near full fine-tuning quality with far fewer trainable parameters.
How much memory does PEFT save?
In one example, LoRA trains just 0.12 percent of a 3B model's weights. A full-model checkpoint of 11 GB shrinks to a 19 MB adapter file.
Can I use PEFT with any model?
It supports many Hugging Face transformer and diffusion models. Check the docs for the supported model list.
Are adapters as good as full fine-tuning?
For most tasks, LoRA matches full fine-tuning closely. Full tuning still wins for very large domain shifts.
Conclusion
PEFT puts fine-tuning within reach of small teams. Training a tiny adapter instead of a whole model cuts memory, storage, and cost, with results close to full fine-tuning. Start with LoRA on a model you already use, save the small adapter file, and share or merge it when you are done. That is the practical way to customize large models.
Keep exploring
Similar AI tools
Ranked by the job they do, their capabilities, and how closely they fit this workflow.