UER-Py

Visit Tool

UER-py is an open-source framework for pre-training and fine-tuning NLP models in PyTorch. It includes a model zoo with various pre-trained models and supports research extensibility.

Claim this tool

1View

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is UER-py?

UER-py (Universal Encoder Representations) is an open-source framework designed for pre-training on general-domain corpora and fine-tuning on downstream NLP tasks using PyTorch. It emphasizes model modularity, allowing users to combine various embedding, encoder, decoder, and target modules to construct custom pre-training models. The toolkit supports CPU, single GPU, and distributed training modes, making it versatile for different computational environments. UER-py also provides a comprehensive model zoo with pre-trained models of diverse properties, facilitating their direct use in various applications. It has been tested for reproducibility against original implementations of models like BERT, GPT-2, ELMo, and T5, and offers solutions for numerous NLP competitions.

Best used for

Ideal for developers and professors who need to pre-train custom NLP models, fine-tune existing models for specific downstream tasks, and conduct research on language model architectures. Especially valuable for those requiring a modular framework that supports distributed training and offers a wide range of pre-trained models.

Common actions

pre-train models

fine-tune models

develop NLP applications

research language models

workflowsopen-sourcelow-code/no-codecollaborationdeepfakeautomated workflowface swappinggithub copilot"AI Agents"

Capabilities

Key features

Model modularity
Distributed training
Pre-trained model zoo
Reproducibility
Downstream task support
Feature extractor
Text generation

Target Audience

developerprofessor

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What kind of pre-trained models are available in the UER-py model zoo?

The UER-py model zoo contains a variety of pre-trained models based on different corpora, encoders, and targets. These models are designed to be directly loadable and usable within the UER-py framework, offering flexibility for various NLP tasks and research needs.

Does UER-py support distributed training for large models?

Yes, UER-py supports multiple training modes, including CPU mode, single GPU mode, and distributed training mode. This allows users to scale their pre-training and fine-tuning efforts for larger models and datasets across multiple GPUs or machines.

How does UER-py ensure reproducibility of pre-training model implementations?

UER-py has been rigorously tested across many datasets to match the performance of original pre-training model implementations such as BERT, GPT-2, ELMo, and T5. This focus on reproducibility helps researchers and developers trust the framework's results.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Also listed in

This tool also appears in

Research & Education › Academic Research AI Agents & Automation › AI Frameworks & Infra Research & Education › Scientific Computing

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce