Home / Tools / PEFT

PEFT

Efficient fine-tuning with LoRA, QLoRA, and adapters

Visit PEFT
PEFT preview
github.com Open Source

Upvotes

0

Pricing

Open Source

Category

Coding Tools

In the database since

Oct 2026

Overview

About PEFT

PEFT is Hugging Face's open-source library for parameter-efficient fine-tuning of large models. Fine-tuning a whole model updates billions of weights, which costs a lot of memory and storage. PEFT instead trains only a small set of extra parameters, called adapters, while the base model stays frozen. That makes customizing big models for your own tasks cheap and practical, even on modest hardware. It works with Transformers, Diffusers, and other Hugging Face libraries, and it fits into existing training code with small changes.

Key Features

LoRA

LoRA trains low-rank adapter matrices instead of full weights. You get full-model quality at a fraction of the compute and memory.

QLoRA

QLoRA combines LoRA with 4-bit quantization for even bigger savings. Huge models fit on modest GPUs.

Many Methods

PEFT includes adapters, prefix tuning, and more techniques. One API covers them all. Try different methods without rewriting your code.

Hugging Face Native

It plugs into Transformers, Diffusers, and TRL seamlessly. Your existing code barely changes. Adapters save as small, shareable files.

Mergeable Adapters

Trained adapters can merge back into the base model. Deployment stays simple with one file.

Frequently Asked Questions (FAQ)

How do I install PEFT?

Install it with pip install peft, then wrap your model with get_peft_model and a config like LoraConfig. The quickstart shows a full working example.

What is LoRA?

LoRA (Low-Rank Adaptation) trains tiny weight matrices that plug into a frozen model. It gives near full fine-tuning quality with far fewer trainable parameters.

How much memory does PEFT save?

In one example, LoRA trains just 0.12 percent of a 3B model's weights. A full-model checkpoint of 11 GB shrinks to a 19 MB adapter file.

Can I use PEFT with any model?

It supports many Hugging Face transformer and diffusion models. Check the docs for the supported model list.

Are adapters as good as full fine-tuning?

For most tasks, LoRA matches full fine-tuning closely. Full tuning still wins for very large domain shifts.

Conclusion

PEFT puts fine-tuning within reach of small teams. Training a tiny adapter instead of a whole model cuts memory, storage, and cost, with results close to full fine-tuning. Start with LoRA on a model you already use, save the small adapter file, and share or merge it when you are done. That is the practical way to customize large models.

Newsletter

Tools like this change fast

New Coding Tools tools launch every week — the newsletter keeps you ahead of what's worth trying.

Keep exploring

Similar AI tools

Ranked by the job they do, their capabilities, and how closely they fit this workflow.