Shypd.aiThe largest open index of AI toolsServing is an open-source framework for deploying machine learning models, offering high-performance and flexible online inference services. It supports various protocols and hardware, integrating with PaddlePaddle for industrial-grade deployments.
Trending
About
Paddle Serving, built on the PaddlePaddle deep learning framework, provides a high-performance and flexible industrial-grade online inference service for machine learning models. It supports multiple protocols like RESTful, gRPC, and bRPC, and is compatible with diverse heterogeneous hardware and operating systems. The framework features both a high-performance C++ Serving component, optimized for high throughput and low latency using the bRPC network framework, and a user-friendly Python Pipeline for easy deployment. It also includes an asynchronous pipeline inference framework with capabilities like multi-model composition, asynchronous scheduling, concurrent inference, dynamic batching, and multi-card/multi-stream inference. Paddle Serving integrates acceleration libraries like Intel MKLDNN and Nvidia TensorRT, and offers solutions for secure model deployment, including encryption and authentication.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
