Shypd.aiThe largest open index of AI toolsTurboTransformers is an open-source runtime environment for fast transformer inference on CPU and GPU. It supports both transformer encoders and decoders, and handles variable length inputs without offline tuning.
Trending
About
TurboTransformers is an open-source, fast, and user-friendly runtime environment designed for transformer inference on both CPU and GPU. Developed by WeChat AI, it supports various transformer models including BERT, ALBERT, GPT2, and Decoders. A key feature is its ability to handle variable length inputs without requiring time-consuming offline tuning, allowing for real-time changes in batch size and sequence length. It offers excellent CPU/GPU performance and includes smart batching to minimize zero-padding overhead for requests of different lengths. TurboTransformers provides both Python and C++ APIs, and can be integrated as a plugin for PyTorch, enabling end-to-end acceleration with just a few lines of code. It has been successfully applied in Tencent's online BERT service scenarios, demonstrating significant acceleration for services like WeChat FAQ and QQ recommendation systems.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
