Tpu-Mlir

Visit Tool

TPU-MLIR is an open-source machine learning compiler that converts pre-trained neural networks into binary files for efficient operation on Sophgo TPUs, supporting frameworks like PyTorch, ONNX, TFLite, and Caffe.

Claim this tool

1View

At a glance

Pricing

Open Source

Free tier

Yes

API

Skill level

Technical

About

What is tpu-mlir?

TPU-MLIR is an open-source machine learning compiler built on MLIR, specifically designed for Sophgo TPUs. It provides a comprehensive toolchain to convert pre-trained neural networks from various deep learning frameworks, including PyTorch, ONNX, TFLite, and Caffe, into optimized binary files (bmodel) that can run efficiently on TPUs. The project also supports compiling HuggingFace LLM models, with current support for Qwen2 and Llama series, and plans for more. It offers tools for model transformation, deployment, and calibration, enabling users to convert models to different quantization types like F16 and INT8, and provides auxiliary tools for model inference and bmodel manipulation.

Best used for

Ideal for developers who need to convert pre-trained neural networks from frameworks like PyTorch or ONNX into optimized binary files for Sophgo TPUs, and compile HuggingFace LLM models. Especially valuable for achieving efficient hardware acceleration and deploying AI models on specialized hardware.

Common actions

compile machine learning models

optimize neural networks

deploy AI models

convert deep learning frameworks

quantize models

deepfakeopen-sourcecollaborationautomated workflowlow-code/no-code"AI Agents"github copilotworkflowsface swapping

Capabilities

Key features

MLIR-based compiler
Sophgo TPU optimization
Multi-framework support
LLM model compilation
Quantization (F16, INT8)
Model inference tools
Bmodel manipulation

Target Audience

developer

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What deep learning frameworks does TPU-MLIR support for model conversion?

TPU-MLIR supports converting pre-trained neural networks from PyTorch, ONNX, TFLite, and Caffe. Models from other frameworks can be converted to ONNX first. It also supports specific HuggingFace LLM models like Qwen2 and Llama series.

How do I get started with TPU-MLIR?

You can start by installing Docker and pulling the sophgo/tpuc_dev image. Then, you can either use the prebuilt Python package via pip or compile the source code within the Docker environment. Detailed build and usage instructions are provided in the documentation.

Can TPU-MLIR quantize models for better performance?

Yes, TPU-MLIR supports quantization to F16 and INT8 types. For INT8 quantization, you need to run a calibration process using a dataset to generate a calibration table, which is then used during model deployment to create an optimized INT8 bmodel.

Trending

Subcategories trending in Coding & Development

Code Assistants DevOps & Infrastructure No-Code / Low-Code Testing & QA Backend & APIs Prompt Engineering

Trending

Also listed in

This tool also appears in

AI Agents & Automation › AI Frameworks & Infra

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce