Shypd.aiThe largest open index of AI toolsSGLang is a high-performance serving framework for large language models and multimodal models. It provides efficient inference across various hardware setups, from single GPUs to distributed clusters.
Trending
Also listed in
About
SGLang is a high-performance serving framework designed for large language models and multimodal models, focusing on low-latency and high-throughput inference. It supports a wide range of hardware, including NVIDIA, AMD, Intel, Google TPUs, and Ascend NPUs, and is compatible with most Hugging Face models and OpenAI APIs. Key features include RadixAttention for prefix caching, a zero-overhead CPU scheduler, prefill-decode disaggregation, speculative decoding, continuous batching, paged attention, and various parallelism techniques. SGLang also supports structured outputs, chunked prefill, quantization, and multi-LoRA batching. It is an open-source project with an active community, adopted by leading enterprises and institutions, and serves as a proven rollout backend for training frontier models.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
