Shypd.aiThe largest open index of AI toolsauto-round is an open-source quantization toolkit for Large Language Models (LLMs) and Vision-Language Models (VLMs). It achieves high accuracy at ultra-low bit widths (2–4 bits) with minimal tuning and broad hardware compatibility.
Trending
About
auto-round is an advanced, open-source quantization toolkit developed by Intel, designed for optimizing Large Language Models (LLMs) and Vision-Language Models (VLMs). It excels at achieving high accuracy even at ultra-low bit widths (2–4 bits) with minimal tuning, leveraging sign-gradient descent. The toolkit offers broad hardware compatibility, supporting CPU, XPU, and CUDA, and integrates seamlessly with popular ecosystems like Transformers, vLLM, and SGLang. Key features include support for multiple export formats (AutoRound, AutoAWQ, AutoGPTQ, GGUF), fast mixed bits/datatypes scheme generation, and affordable quantization costs, allowing 7B models to be quantized in about 10 minutes on a single GPU. It also supports over 10 VLMs and provides advanced utilities like multiple GPU quantization and various calibration datasets.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
