Shypd.aiThe largest open index of AI toolspytorch-openai-transformer-lm is an open-source Coding & Development tool that provides a PyTorch implementation of OpenAI's finetuned transformer language model. It includes a script to import pre-trained weights, enabling researchers and developers to work with the model.
Trending
Also listed in
About
pytorch-openai-transformer-lm offers a PyTorch implementation of OpenAI's finetuned transformer language model, based on the paper "Improving Language Understanding by Generative Pre-Training." This tool includes a script to import the weights pre-trained by OpenAI, allowing users to leverage the model within a PyTorch environment. It supports fine-tuning the pre-trained model for classification tasks, with an example provided for the ROCStories Cloze task. The implementation closely follows the original TensorFlow code, including a modified Adam optimization algorithm with fixed weight decay and scheduled learning rate. It provides classes for a full language model with a tied decoder and a classifier head on top of the transformer.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
