Shypd.aiThe largest open index of AI toolsCurator is an open-source data preprocessing and curation toolkit for LLMs, offering GPU-accelerated pipelines for text, images, video, and audio. It scales from laptops to multi-node clusters, significantly speeding up data preparation for AI models.
Trending
About
Curator is a scalable, open-source data preprocessing and curation toolkit developed by NVIDIA NeMo, designed to enhance the training of large language models (LLMs) and other AI models. It provides GPU-accelerated, modular pipelines for various data modalities including text, images, video, and audio. The tool supports a wide range of capabilities such as data deduplication, quality filtering, language detection, aesthetic filtering, NSFW detection, and ASR transcription. Curator is built to scale from individual laptops to multi-node clusters, leveraging NVIDIA RAPIDS™ libraries like cuDF, cuML, and cuGraph, along with Ray, to achieve significant performance improvements and cost reductions compared to CPU-based alternatives. It is a core component of the NVIDIA NeMo software suite, facilitating the entire AI agent lifecycle.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
