Shypd.aiThe largest open index of AI toolsLMM-R1 is an Open Source research tool that extends OpenRLHF to support Large Multimodal Model (LMM) Reinforcement Learning (RL) training. It empowers 3B LMMs with strong reasoning abilities through a two-stage rule-based RL framework.
Trending
About
LMM-R1 is an open-source project designed to enhance the reasoning capabilities of 3B Large Multimodal Models (LMMs) by extending the OpenRLHF framework. It addresses the challenges of limited parameter capacity and scarce high-quality multimodal reasoning data through a novel two-stage rule-based RL approach. The first stage, Foundational Reasoning Enhancement (FRE), builds strong reasoning foundations using text-only data. The second stage, Multimodal Generalization Training (MGT), extends these capabilities to multimodal domains. LMM-R1 supports various LMMs like Qwen2.5-VL, Phi3.5-V, and Phi4-Multimodal, and offers distributed PPO and REINFORCE++/RLOO implementations based on Ray, achieving significant speedups. It also integrates with vLLM for accelerated generation, FlashAttention2, and supports QLoRA/LoRA for efficient fine-tuning.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
