by Michele Laurelli
Parameter-efficient fine-tuning adding trainable low-rank matrices to frozen weights.
Adds small trainable matrices A and B where ΔW = BA. Reduces trainable parameters to 0.1-1% while maintaining performance. Enables efficient adapter-based fine-tuning.
Fine-tuning LLMs with LoRA
Adapter modules
Multi-task learning