Shypd.aiThe largest open index of AI toolsJittorLLMs is an open-source library for large language model inference, offering high performance and low configuration requirements. It enables local deployment of large models even on machines with limited resources, including those without a dedicated GPU.
Trending
About
JittorLLMs is a large language model inference library designed for high performance, low configuration requirements, and excellent Chinese language support. A key differentiator is its ability to run large models on machines with as little as 2GB RAM and no dedicated GPU, making local deployment accessible to a wider range of users. The library supports a variety of popular models including ChatGLM, Peng Cheng PanGu, ChatRWKV, LLaMA/LLaMA2, MOSS, and Atom7B, with plans to integrate more domestic models. It boasts significant speed improvements, reducing model loading times by 40% and boosting computational performance by over 20% compared to similar frameworks, thanks to zero-copy technology and automatic meta-operator compilation. JittorLLMs also offers high portability, allowing users to migrate models to various heterogeneous computing devices and environments by simply installing Jittor-version Torch (JTorch) without code modification. It includes features for memory optimization, such as dynamic swap technology, to manage memory and VRAM usage efficiently.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
