Shypd.aiThe largest open index of AI toolsprime-diloco is a framework for efficient, globally distributed training of AI models over the internet. It offers fault-tolerant training, asynchronous checkpointing, and custom Int8 All-Reduce kernels for optimized performance.
Trending
About
prime-diloco, previously known as ZeroBand, is an open-source framework designed for efficient and globally distributed training of AI models across the internet. It introduces an ElasticDeviceMesh for fault-tolerant training, allowing dynamic resizing of global process groups when nodes join or leave, unlike standard DeviceMesh implementations. The framework also features asynchronous distributed checkpointing to minimize blocking time during training, live checkpoint recovery for nodes joining mid-training, and a custom Int8 All-Reduce kernel for significant payload size reduction without impacting loss curves. Additionally, it maximizes bandwidth utilization through sharding and VPN technology, and implements PyTorch FSDP2 / DTensor ZeRO-3 for memory optimization and CPU off-loading for its optimizer.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
