Shypd.aiThe largest open index of AI toolsModernBERT is an AI Agents & Automation tool that modernizes BERT through architectural changes and scaling. It introduces FlexBERT, a modular approach to encoder building blocks, and heavily relies on YAML configuration files.
Trending
About
ModernBERT is a research repository focused on bringing BERT into modernity through both architectural changes and scaling. It introduces FlexBERT, a modular approach to encoder building blocks, and utilizes YAML configuration files for model building. The codebase builds upon MosaicBERT, incorporating Flash Attention 2 for efficiency. This repository provides the tools and experiments for pre-training and GLUE evaluations of ModernBERT models. It also includes examples for training retrieval models, both dense models based on Sentence Transformers and ColBERT models via the PyLate library. The project is a collaboration between Answer.AI, LightOn, and other contributors, offering a robust framework for advanced natural language processing research and development.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
