Shypd.aiThe largest open index of AI toolsHivemind is an open-source deep learning tool that enables decentralized training of PyTorch models. It allows models to be trained across thousands of volunteer computers globally, supporting fault-tolerant backpropagation and decentralized parameter averaging.
Trending
About
Hivemind is a PyTorch library designed for decentralized deep learning, facilitating the training of large models across a vast network of computers, including those from different universities, companies, and volunteers. Its core features include distributed training without a master node, leveraging a Distributed Hash Table for network connectivity. The library ensures fault-tolerant backpropagation, allowing training to continue even if some nodes become unresponsive. It also implements decentralized parameter averaging, enabling iterative aggregation of updates from multiple workers without requiring network-wide synchronization. Furthermore, Hivemind supports training neural networks of arbitrary size by distributing parts of their layers using Decentralized Mixture-of-Experts. It is compatible with Python 3.8+ and PyTorch 1.9.0+ and offers integration with PyTorch Lightning for adapting existing pipelines to slow, unreliable networks.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
