Coding & Development

Sonnet is a powerful open-source library built on top of TensorFlow 2, specifically designed by DeepMind researchers to provide simple, composable abstractions for machine learning research. It facilitates the construction of neural networks for various purposes, including supervised learning, unsupervised learning, and reinforcement learning. The library centers around the `snt.Module` concept, allowing users to define custom modules or utilize many predefined ones like `snt.Linear` and `snt.Conv2D`. Sonnet is unopinionated about training frameworks, encouraging users to build their own or adopt existing ones. It emphasizes clarity and focus in its codebase, making it easy to understand and extend. Key features include support for TensorFlow checkpointing and saved models, enabling robust serialization and deployment of models.

Quantum Elements offers an AI-native quantum development platform designed to accelerate the building, optimization, and launching of quantum applications. The platform integrates AI-powered quantum simulation, automated calibration, and circuit optimization to enhance the development process. Key features include a noise-aware quantum simulation environment, an AI-powered quantum assistant, and real-time error suppression, making it suitable for both researchers and engineers. It supports hardware-agnostic execution, providing flexibility for various quantum computing environments. The platform aims to simplify complex quantum development tasks, allowing users to focus on innovation rather than intricate technical challenges.

Meta AI Demos is a platform designed to provide users with direct access to experimental AI demonstrations and research developed by Meta. It serves as a showcase for the company's latest advancements in artificial intelligence, allowing individuals to interact with and explore various AI models and applications. The platform is ideal for AI researchers, developers, and technology enthusiasts who wish to understand the practical capabilities and potential of new AI technologies. By offering hands-on experiences, Meta AI Demos facilitates learning and exploration of the evolving AI landscape, directly from one of the leading research institutions.

spark-deep-learning provides Deep Learning Pipelines for Apache Spark, focusing on distributed deep learning training jobs using Horovod. While the open-source version primarily supports local development, it integrates seamlessly with Databricks Runtime for Machine Learning to launch Horovod jobs as distributed Spark jobs. This allows data scientists and engineers to manage cluster setup and scale their deep learning models efficiently. The library offers functionalities for running training code across multiple processes, with options to specify the number of parallel processes for optimal resource utilization on GPU or CPU clusters. It simplifies the deployment and execution of complex deep learning tasks within the Spark ecosystem.

stocksight is an open-source stock market analysis and prediction software that leverages Elasticsearch to store and process data from Twitter and news headlines. It performs natural language processing and sentiment analysis on this data to determine how public sentiment might affect stock prices. The tool can be used to analyze emotions related to specific stocks or broader topics. It requires Python 3.x, Elasticsearch 5.x, and Kibana 5.x, and can be installed locally or via Docker. Users can configure keywords for Twitter mining, follow links in tweets and news headlines for deeper sentiment analysis, and integrate stock price data. stocksight provides a comprehensive framework for data scientists and financial analysts to explore the relationship between public sentiment and market trends.

Rhombus offers a cloud-based video management system (VMS) designed to streamline physical security operations. It integrates security cameras, access control, and sensors into a single, user-friendly platform with AI analytics and real-time alerts. The system provides remote visibility into security operations through a console, supporting live monitoring, easy sharing, and smart search capabilities. Users can scale to unlimited cameras, doors, users, devices, and locations, all managed from an intuitive cloud VMS. Rhombus emphasizes proactive security with flexible, real-time alerts and automated workflows based on AI threat detection, ensuring secure and efficient facility management.

EvalsOne was a platform designed for evaluating and optimizing generative AI applications, supporting all stages of LLMOps from development to production. Key features included an intuitive interface, comprehensive functionality, automated insights, and A/B testing. It integrated with various cloud services, local models, orchestration tools, and AI bot APIs. However, EvalsOne has been sunset, and its services are no longer available. Users seeking similar functionalities are now directed to ConsoleX at consolex.ai.

Tambo is an open-source generative UI toolkit for React, designed to help developers build AI agents that can render and interact with user interfaces. It provides a full-stack solution including a React SDK and a backend for conversation state and agent execution. Tambo supports various LLM providers like OpenAI, Anthropic, and Google Gemini, allowing developers to bring their own API keys. Key features include streaming infrastructure for props, cancellation, error recovery, and reconnection, as well as options for Tambo Cloud or self-hosting via Docker. It enables the creation of both generative components that render once and interactable components that persist and update based on user requests, making it ideal for dynamic and adaptive applications.

Time-LLM is an official implementation of a reprogramming framework designed to repurpose Large Language Models (LLMs) for general time series forecasting, while keeping the backbone language models intact. It posits that time series analysis can be effectively treated as a language task for off-the-shelf LLMs. The framework consists of two main components: reprogramming input time series into text prototype representations suitable for LLMs, and augmenting input context with declarative prompts, including domain expert knowledge and task instructions, to guide LLM reasoning. The tool supports various LLMs, including Llama-7B, GPT-2, and BERT, and has been adopted for solar, wind, and weather forecasting by XiMou Optimization Technology Co., Ltd. (XMO).

torch.rb provides deep learning capabilities for Ruby developers, leveraging the power of LibTorch. It allows users to create and manipulate tensors, perform various operations, and build neural networks directly within the Ruby environment. The library closely follows the PyTorch API, with minor adjustments to be more Ruby-like, making it easier for developers familiar with PyTorch to transition. It supports tasks such as image classification, collaborative filtering, and generative adversarial networks, and integrates with TorchVision, TorchText, and TorchAudio for specialized computer vision, NLP, and audio tasks. Performance can be significantly enhanced on GPUs, with support for CUDA on Linux and Metal Performance Shaders (MPS) on Mac.

TonY is an open-source framework designed to natively execute deep learning frameworks such as TensorFlow, PyTorch, MXNet, and Horovod on Apache Hadoop. It enables users to run both single-node and distributed training jobs as a Hadoop application, providing a robust and flexible environment for machine learning workflows. Key features include compatibility with Hadoop 2.6.0+ (CDH5.11.0+) and support for GPU isolation with newer Hadoop versions. Users can launch deep learning jobs either by utilizing a zipped Python virtual environment or by leveraging Docker containers within their Hadoop cluster. TonY offers extensive configuration options via XML files or command-line arguments, allowing for fine-grained control over job parameters like worker instances, memory, and GPU allocation. It also includes examples for distributed MNIST with various frameworks and integration with Google Cloud Platform and Azkaban.

TPU-MLIR is an open-source machine learning compiler built on MLIR, specifically designed for Sophgo TPUs. It provides a comprehensive toolchain to convert pre-trained neural networks from various deep learning frameworks, including PyTorch, ONNX, TFLite, and Caffe, into optimized binary files (bmodel) that can run efficiently on TPUs. The project also supports compiling HuggingFace LLM models, with current support for Qwen2 and Llama series, and plans for more. It offers tools for model transformation, deployment, and calibration, enabling users to convert models to different quantization types like F16 and INT8, and provides auxiliary tools for model inference and bmodel manipulation.

Rubra is an open-source tool designed for developers to create and test AI assistants locally. It leverages large language models, providing a private and cost-effective environment for AI development without relying on external API calls or token usage. This approach ensures data privacy and significantly reduces operational costs associated with cloud-based solutions. Rubra facilitates the creation of AI-powered applications, offering a ChatGPT-like experience for local AI development, making it an ideal choice for those seeking to build and experiment with AI models in a controlled, offline setting.

vllm-ascend is a community-maintained hardware plugin designed to integrate vLLM with Ascend NPUs, allowing for seamless execution of large language models on Ascend hardware. It adheres to a hardware-pluggable interface, decoupling the integration of Ascend NPUs with vLLM. This plugin supports various open-source models, including Transformer-like, Mixture-of-Experts (MoE), Embedding, and Multi-modal LLMs. It is the recommended approach for supporting the Ascend backend within the vLLM community, enhancing performance for fine-tuning, evaluation, reinforcement learning (RL), and deployment scenarios. The project provides detailed documentation for getting started and contributing, with active development branches and regular releases.

XLA (Accelerated Linear Algebra) is an open-source machine learning compiler designed to boost the performance of ML models across various hardware platforms. It takes models developed in popular frameworks such as PyTorch, TensorFlow, and JAX, and optimizes them for high-performance execution on GPUs, CPUs, and specialized ML accelerators. This compiler is a critical tool for developers and researchers looking to achieve greater speed and efficiency in their machine learning workloads. While users can leverage XLA through their ML frameworks, the repository itself is primarily intended for XLA contributors and integrators who wish to develop the compiler or add support for new ML frontends and hardware backends.

Guardian-cli is an enterprise-grade AI-powered penetration testing automation framework designed for security professionals. It integrates multiple AI providers like OpenAI GPT-4, Claude, Google Gemini, and OpenRouter with a robust arsenal of 19 battle-tested security tools. The platform delivers intelligent, adaptive security assessments with comprehensive evidence capture, including execution traceability, complete command history, and raw evidence storage. Guardian-cli features a multi-agent architecture for strategic decision-making and adaptive testing, along with false positive filtering. It supports various reporting formats (Markdown, HTML, JSON) with executive summaries, technical deep-dives, and AI decision traces for full transparency. The tool also includes security and compliance features like scope validation, audit logging, human-in-the-loop confirmation, and a safe mode to prevent destructive actions.

LLMTools is an open-source Python library designed for efficiently running and finetuning Large Language Models (LLMs) in low-resource environments, specifically on consumer-grade GPUs. It features advanced finetuning capabilities in 2-bit, 3-bit, and 4-bit precision, leveraging the innovative ModuLoRA algorithm. The library provides an easy-to-use Python API for various tasks including quantization, inference, and finetuning. A key differentiator is its modular support for multiple LLMs, quantizers, and optimization algorithms, allowing for flexibility and integration with the HuggingFace Hub for sharing finetuned models. Developed as a research project at Cornell University, LLMTools is based on cutting-edge publications like ModuLoRA and QuIP, making it a valuable tool for researchers and developers working with LLMs.

The LLM-Engineers-Handbook is an official repository and practical guide for building end-to-end LLM-based systems, developed by Paul Iusztin and Maxime Labonne. It covers essential aspects from data collection and generation to LLM training pipelines, simple RAG systems, and production-ready AWS deployment. The handbook emphasizes LLMOps best practices, including comprehensive monitoring, testing, and evaluation frameworks. It details the use of various tools and cloud services like HuggingFace, Comet ML, Opik, ZenML, AWS, MongoDB, Qdrant, and GitHub Actions. The repository provides actively maintained code, installation instructions, and guidance on setting up local development and cloud deployment environments.

MiniChain is a lightweight Python library designed for coding with large language models, offering a streamlined approach to prompt chaining. It enables developers to annotate Python functions for direct interaction with various language models and provides a visual graph of all calls for enhanced debugging and error handling. The library supports prompt engineering through Jinja templates, separating prompt text from code for better organization. MiniChain integrates with backends like OpenAI, Hugging Face, Google Search, and Python, and supports popular approaches such as Retrieval-Augmented QA, Chat with memory, and Chain-of-Thought. It also features a built-in visualization system using Gradio for interactive debugging and typed prompts for structured output generation.

DeepAudit is an open-source, multi-agent AI system designed to make code vulnerability detection and auditing accessible. It simulates the thought process of security experts through a collaborative architecture involving Orchestrator, Recon, Analysis, and Verification agents. This system aims to overcome common issues with traditional SAST tools, such as high false-positive rates, blind spots in business logic, and a lack of verification methods. Users can import projects, and DeepAudit will automatically identify tech stacks, analyze risks, generate scripts, perform sandbox verification, and produce professional audit reports. It supports Ollama for private deployment, ensuring data privacy, and has successfully identified numerous CVEs and GHSA security advisories.

MusPy is an open-source Python library designed to streamline the development of symbolic music generation systems. It offers a comprehensive suite of tools for various stages of the music generation pipeline, from data collection and preprocessing to model creation, training, and evaluation. Key features include a robust dataset management system with interfaces to PyTorch and TensorFlow, and extensive data I/O capabilities for common symbolic music formats like MIDI, MusicXML, and ABC. MusPy also provides implementations of various music representations, such as pitch-based, event-based, piano-roll, and note-based, catering to diverse generation approaches. Additionally, it includes model evaluation tools for audio rendering, score and piano-roll visualizations, and objective metrics, making it a valuable resource for researchers and developers in music AI.

NVIDIA Model Optimizer is an open-source library designed to accelerate deep learning models through various state-of-the-art optimization techniques. It supports quantization, pruning, distillation, speculative decoding, and sparsity to compress models and enhance inference speed. The tool accepts Hugging Face, PyTorch, or ONNX models as input and provides Python APIs for composing optimization techniques. Optimized checkpoints can be seamlessly exported for deployment in frameworks like SGLang, TensorRT-LLM, TensorRT, and vLLM, making it a crucial component within the NVIDIA AI software ecosystem for efficient model deployment.

text-generation-webui is a comprehensive, open-source local LLM interface designed for a wide range of AI tasks including text generation, vision capabilities, tool-calling, and model training. It provides both a user-friendly UI and an API, ensuring 100% offline and private operation with zero telemetry or external requests. The tool supports various backends like llama.cpp, Transformers, and ExLlamaV3, and is compatible with GGUF models. Key features include instruct and chat modes, multimodal vision for image understanding, file attachments for content analysis, and the ability to fine-tune LoRAs. It also offers image generation with diffusers models and supports 4-bit/8-bit quantization, making it a versatile solution for local AI deployment.

GPTQModel is a comprehensive toolkit designed for the quantization and compression of Large Language Models (LLMs). It significantly reduces model size and improves inference speed by supporting hardware acceleration across a wide range of platforms, including NVIDIA CUDA, AMD ROCm, Intel XPU, and Intel/AMD/Apple CPUs. The toolkit seamlessly integrates with leading LLM frameworks such as Hugging Face (HF), vLLM, and SGLang, making it versatile for various deployment scenarios. Key features include support for multiple quantization methods like ParoQuant, GGUF, FP8, and EXL3, along with advanced optimizations for MoE models and improved memory usage during quantization. It also boasts JIT-compiled CUDA kernels for efficiency and continuous updates for new model support and performance enhancements.