What machine learning frameworks does Simple TensorFlow Serving support?
Simple TensorFlow Serving supports a wide range of machine learning frameworks, including TensorFlow, MXNet, PyTorch, Caffe2, CNTK, ONNX, H2o, Scikit-learn, XGBoost, PMML, and Spark MLlib. This broad compatibility allows users to deploy models from various ecosystems using a single serving solution.
Can I deploy multiple models and versions simultaneously?
Yes, Simple TensorFlow Serving is designed to serve multiple models and multiple versions of these models concurrently. It automatically detects and reloads the latest files for adding or removing model versions, providing flexibility and control over your deployed models.
Does Simple TensorFlow Serving support GPU acceleration for inference?
Yes, the tool supports inference with accelerated GPUs. Users can configure GPU options via command-line parameters or within the model configuration file, and specific Docker images are available for GPU-enabled deployments to leverage hardware acceleration.
How does Simple TensorFlow Serving handle client generation?
Simple TensorFlow Serving can automatically generate clients in various programming languages like Python, Bash, Golang, and JavaScript. This feature allows users to interact with their deployed models without writing custom client-side code, streamlining the integration process.
Is there a way to secure the deployed model endpoints?
Yes, Simple TensorFlow Serving offers secure authentication with configurable basic auth, denying anonymous requests. It also supports TSL/SSL encryption, allowing users to generate self-signed secret files for testing and secure communication with their model endpoints.