Shypd.aiThe largest open index of AI toolsFastDeploy is a high-performance inference and deployment toolkit for large language models (LLMs) and vision language models (VLMs). It offers production-grade deployment solutions with advanced acceleration techniques and multi-hardware support.
Trending
About
FastDeploy is an open-source, high-performance inference and deployment toolkit built on PaddlePaddle, designed for Large Language Models (LLMs) and Vision Language Models (VLMs). It provides production-ready deployment solutions with key features like load-balanced PD decomposition, unified KV cache transmission, and compatibility with OpenAI API services and vLLM. The toolkit supports a wide range of quantization formats including W8A16, W8A8, W4A16, W4A8, W2A16, and FP8. It also incorporates advanced acceleration technologies such as speculative decoding, multi-token prediction (MTP), and chunked prefilling. FastDeploy supports various hardware platforms including NVIDIA GPUs, Kunlunxin XPUs, Hygon DCUs, and Intel Gaudi, making it a versatile tool for AI developers and machine learning engineers.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
