Shypd.aiThe largest open index of AI toolsRewardBench is an open-source benchmark designed to evaluate the capabilities and safety of reward models, including those trained with Direct Preference Optimization (DPO). It provides common inference code, dataset formatting, and analysis tools for fair reward model assessment.
Trending
About
RewardBench is an open-source benchmark and evaluation tool specifically designed for assessing the capabilities and safety of reward models, including those utilizing Direct Preference Optimization (DPO). The repository offers common inference code compatible with various reward models such as Starling, PairRM, OpenAssistant, and DPO. It ensures fair evaluation through standardized dataset formatting and testing procedures. Additionally, RewardBench includes robust analysis and visualization tools to help researchers and developers interpret results effectively. It supports quick evaluation of any reward model on any preference set, with features for logging model outputs and accuracy scores, and options for generative models (LLM-as-judge) and DPO models. The platform also facilitates contributing models to a public leaderboard and offers offline ensemble testing.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
