Shypd.aiThe largest open index of AI toolsHALOs is an open-source library for aligning Large Language Models (LLMs) using various human-aware loss functions like DPO, KTO, PPO, and ORPO. It offers modular, extensible, and simple implementations for training and evaluation.
Trending
About
HALOs (Human-Centered Loss Functions) is a Python library designed to facilitate the alignment of Large Language Models (LLMs) with human preferences. It provides extensible implementations of popular alignment methods such as DPO, KTO, PPO, and ORPO. The library emphasizes modularity, separating dataloading, training, and sampling, and extensibility, allowing users to quickly implement custom dataloaders or new alignment losses. HALOs is built for simplicity, making it easy to hack on, and has been tested with LLMs ranging from 1B to 30B parameters. It supports LoRA training, reference logit caching to reduce memory, and integrates with tools like Hydra for configuration and Accelerate for job launching with FSDP. The repository also includes scripts for evaluation with AlpacaEval and LMEval.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
