Shypd.aiThe largest open index of AI toolsLoRA is an open-source library for low-rank adaptation of large language models, designed to reduce trainable parameters and storage requirements. It enables efficient task-switching without introducing inference latency.
Trending
About
LoRA (Low-Rank Adaptation of Large Language Models) is an open-source Python package that provides an implementation of the LoRA technique for fine-tuning large language models. It significantly reduces the number of trainable parameters by learning pairs of rank-decomposition matrices while freezing the original weights. This approach vastly decreases storage requirements for models adapted to specific tasks and allows for efficient task-switching during deployment without adding inference latency. LoRA supports PyTorch models, including those from Hugging Face, and has demonstrated performance comparable to or superior to full fine-tuning on benchmarks like GLUE using models such as RoBERTa and DeBERTa. It also compares favorably to other efficient tuning methods like adapter and prefix-tuning on GPT-2.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
