Shypd.aiThe largest open index of AI toolsStreaming is an AI Frameworks & Infra tool that provides a data streaming library for efficient neural network training. It enables fast and accurate streaming of training data from cloud storage, optimized for multi-node, distributed training.
Trending
Also listed in
About
Streaming is a data streaming library built by MosaicML designed to make training on large datasets from cloud storage as fast, cheap, and scalable as possible. It is specifically optimized for multi-node, distributed training for large models, ensuring correctness, performance, and ease of use. The library supports various data types including images, text, video, and multimodal data, and is compatible with major cloud storage providers like AWS, OCI, GCS, Azure, and any S3 compatible object store. It integrates seamlessly into existing training workflows as a drop-in replacement for PyTorch IterableDataset. Key features include seamless data mixing, true determinism for reproducible training runs, instant mid-epoch resumption, high throughput, and equal convergence compared to local disk solutions.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
