Shypd.aiThe largest open index of AI toolsvllm-ascend is an Open Source & Models tool that provides a community-maintained hardware plugin for running vLLM seamlessly on Ascend NPUs. It enables popular open-source models to run efficiently on Ascend hardware.
Trending
About
vllm-ascend is a community-maintained hardware plugin designed to integrate vLLM with Ascend NPUs, allowing for seamless execution of large language models on Ascend hardware. It adheres to a hardware-pluggable interface, decoupling the integration of Ascend NPUs with vLLM. This plugin supports various open-source models, including Transformer-like, Mixture-of-Experts (MoE), Embedding, and Multi-modal LLMs. It is the recommended approach for supporting the Ascend backend within the vLLM community, enhancing performance for fine-tuning, evaluation, reinforcement learning (RL), and deployment scenarios. The project provides detailed documentation for getting started and contributing, with active development branches and regular releases.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
