Shypd.aiThe largest open index of AI toolsAQLM is an open-source PyTorch implementation for extreme compression of large language models. It uses additive quantization to reduce LLM size and improve inference speed, supporting various LLaMA, Mistral, and Mixtral models.
Trending
About
AQLM is an official PyTorch repository for extreme compression of large language models (LLMs) through additive quantization. This open-source tool significantly reduces the size of LLMs while maintaining or improving accuracy, making them more efficient for deployment and inference. It supports various models from the LLaMA, Mistral, and Mixtral families, offering pre-quantized models and a finetuning algorithm called PV-tuning. AQLM provides inference kernels optimized for both GPU and CPU, with considerable speedups for specific quantization schemes. The repository includes detailed instructions for installation, model quantization, and evaluation, making it a valuable resource for researchers and developers focused on LLM optimization.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
