Xlstm

Visit Tool

xlstm is an Open Source & Models tool that provides the official repository for the xLSTM, an Extended Long Short-Term Memory architecture. It offers code and resources for implementing and experimenting with this novel LSTM variant.

Claim this tool

8Views

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is xlstm?

xlstm is the official GitHub repository for the xLSTM, a new Recurrent Neural Network architecture based on the original LSTM. This tool provides the necessary code and resources for researchers and practitioners to implement and experiment with this novel LSTM variant. xLSTM utilizes Exponential Gating with appropriate normalization and stabilization techniques, along with a new Matrix Memory, to overcome the limitations of traditional LSTMs. It demonstrates promising performance in Language Modeling compared to Transformers or State Space Models. The repository includes a 7B parameter xLSTM Language Model trained on 2.3T tokens, optimized for fast and efficient inference, and offers detailed instructions for installation and usage.

Best used for

Ideal for AI researchers and developers who need to implement and experiment with advanced recurrent neural network architectures, develop novel language models, and benchmark AI models. Especially valuable for those seeking to overcome traditional LSTM limitations with features like Exponential Gating and Matrix Memory.

Common actions

implement recurrent neural networks

develop language models

experiment with LSTM architectures

benchmark AI models

"AI Agents"github copilotface swappingworkflowsopen-sourcecollaborationdeepfakelow-code/no-codeautomated workflow

Capabilities

Key features

Extended Long Short-Term Memory
Exponential Gating
Matrix Memory
7B parameter xLSTM model
PyTorch-based implementation
Triton kernels for inference

Target Audience

professordeveloperresearcher

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What are the key architectural improvements of xLSTM over traditional LSTMs?

xLSTM improves upon traditional LSTMs through Exponential Gating with appropriate normalization and stabilization techniques, and a new Matrix Memory. These innovations help overcome limitations of the original LSTM, leading to promising performance in language modeling tasks compared to other state-of-the-art models.

What are the hardware requirements for using the xLSTM Large 7B model?

For optimal performance with the xLSTM Large 7B model, it is recommended to use NVIDIA GPUs with Compute Capability >= 8.0 for the sLSTM CUDA kernels. The Triton kernels should also run on AMD GPUs. For other platforms like Apple Metal, native PyTorch implementations are recommended.

How can I integrate xLSTM into an existing project or use it for non-language applications?

For non-language applications or integration into other architectures, you can use the xLSTMBlockStack, which functions similarly to a stack of Transformer blocks but utilizes xLSTM blocks. For token-based applications like language modeling, the xLSTMLMModel wrapper is provided.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Also listed in

This tool also appears in

Research & Education › Academic Research AI Agents & Automation › AI Frameworks & Infra

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce