低秩微调
原名：peft
Fine-tune large LLMs with LoRA on limited GPU memory.
- 分类
- 开发提效
- 版本
- v1.0.0
- 作者
- 弈韬(@ra1nzzz)
- 下载
- 1
- 收藏
- 0
- 发布
- 2026-08-25
- 更新
- 2026-09-14
- TRACE 评分
- 3.2 / 5
内容概览
Fine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods. Use PEFT/LoRA when: - Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100) - Need to train <1% parameters (6MB adapters vs 14GB full model) - Want fast iteration with multiple task-specific adapters - Deploying multiple fine-tuned variants from one base model Use QLoRA (PEFT + quantization) when: - Fine-tuning 70B models on single 24GB GPU - Memory is the primary constraint - Can accept 5% quality trade-off vs full fine-tuning Use full fine-tuning instead when: - Training small models (<1B parameters) - Need maximum quality and have compute budget - Significant domain shift requires updating all weights Rank Trainable Params Memory Quality Use Case ------ ----------------- -------- --------- ---------- 4 3M Minimal Lower Simple tasks, prototyping 8 7M Low Good Recommended starting point 16 14M…