Shypd.aiThe largest open index of AI toolsGRPO-Zero is an open-source Code Assistants tool that implements DeepSeek R1's GRPO algorithm from scratch. It is designed for training large language models with minimal dependencies and low GPU memory usage.
Trending
About
GRPO-Zero is an open-source project that provides a from-scratch implementation of DeepSeek R1's Group Relative Policy Optimization (GRPO) algorithm. This tool is specifically designed for training large language models using reinforcement learning, emphasizing minimal dependencies and efficient GPU memory usage. It supports training on GPUs with limited VRAM, such as a 24GB RTX 4090, by offloading the optimizer to the CPU, incurring only a small overhead. Key improvements over the original GRPO include token-level policy gradient loss, removal of KL Divergence for reduced GPU memory, and optional overlong episode filtering to stabilize training. The project demonstrates its capabilities by training Qwen2.5 models on a CountDown task, where the model learns to generate mathematical expressions and reasoning.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
