Jiwer

Visit Tool

JiWER is an open-source Python package for evaluating speech-to-text systems. It quickly calculates word error rate (WER), character error rate (CER), and other similarity measures using minimum-edit distance.

Claim this tool

1View

At a glance

Pricing

Open Source

Free tier

Yes

API

Yes

Skill level

Technical

About

What is jiwer?

JiWER is a simple and fast Python package designed for evaluating automatic speech recognition (ASR) systems. It supports several key similarity measures, including word error rate (WER), match error rate (MER), word information lost (WIL), word information preserved (WIP), and character error rate (CER). These measures are computed efficiently using the minimum-edit distance algorithm, powered by the high-performance RapidFuzz library which leverages C++ for speed. The package also defines specific behaviors for empty reference and hypothesis pairs, addressing potential division-by-zero issues and allowing for testing models on silent audio. JiWER is released under the Apache License, Version 2.0, making it a robust and accessible tool for developers working with speech-to-text technologies.

Best used for

Ideal for developers who need to accurately evaluate the performance of speech-to-text systems, measure text similarity between reference and hypothesis, and benchmark different ASR models. Especially valuable for assessing model behavior with empty references or silent audio.

Common actions

evaluate speech recognition

measure text similarity

benchmark ASR systems

"AI Agents"github copilotface swappingdeepfakeautomated workflowworkflowscollaborationopen-sourcelow-code/no-code

Capabilities

Key features

Calculate word error rate (WER)
Calculate character error rate (CER)
Calculate match error rate (MER)
Compute minimum-edit distance
Handle empty references

Target Audience

developer

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What speech recognition evaluation metrics does JiWER support?

JiWER supports several key metrics for evaluating speech-to-text systems, including Word Error Rate (WER), Match Error Rate (MER), Word Information Lost (WIL), Word Information Preserved (WIP), and Character Error Rate (CER). These provide a comprehensive view of system performance.

How does JiWER handle empty reference or hypothesis strings?

As of version 4.0, JiWER explicitly defines behavior for empty reference and hypothesis pairs. For instance, an empty reference and hypothesis result in zero error, while a non-empty hypothesis with an empty reference counts as insertions. This helps test models for hallucination on silent audio.

What makes JiWER fast for computing error rates?

JiWER achieves its speed by utilizing the RapidFuzz library, which is implemented in C++ under the hood. This allows for efficient calculation of the minimum-edit distance between reference and hypothesis sentences, making it faster than pure Python implementations.

Trending

Subcategories trending in Coding & Development

Open Source & Models Code Assistants DevOps & Infrastructure No-Code / Low-Code Backend & APIs Prompt Engineering

Trending

Also listed in

This tool also appears in

Content & Design › Audio & Music Data & Analytics › Statistical & Scientific

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce