Research & Education

Meshed-Memory Transformer (M²) is an open-source project that provides the reference code for the paper "Meshed-Memory Transformer for Image Captioning" presented at CVPR 2020. This tool is designed for researchers and developers working in computer vision and natural language processing. It allows users to set up a conda environment, download necessary data like COCO annotations and detection features, and then evaluate or train their own image captioning models. The repository includes scripts for both testing and training, with configurable arguments for batch size, number of memory vectors, and learning rate scheduling. It requires Python 3.6 and specific data preparation steps to function correctly.

DeepResearcher is an open-source framework designed to scale deep research by training LLM-based agents using reinforcement learning in real-world web environments. This comprehensive tool facilitates end-to-end training, allowing agents to engage in authentic web search interactions. Qualitative analysis of the framework reveals emergent cognitive behaviors, including the ability to formulate plans, cross-validate information from multiple sources, self-reflect to redirect research, and maintain honesty when definitive answers are unavailable. DeepResearcher demonstrates significant performance improvements over prompt engineering and RAG-based baselines, emphasizing the critical role of end-to-end training in real-world settings for developing robust research capabilities.

mlbookcamp-code is a GitHub repository offering comprehensive code examples and supplementary materials directly from the Machine Learning Bookcamp book. It covers a wide range of machine learning topics, from regression and classification to neural networks, deployment, and serverless deep learning. The repository also provides code for setting up environments, an introduction to Python, NumPy, and Pandas. It serves as a practical companion to the book, allowing users to explore and implement machine learning concepts. Additionally, it links to the Machine Learning Zoomcamp, a free online course based on the book, providing further learning opportunities and community support.

DeepLOB-Deep-Convolutional-Neural-Networks-for-Limit-Order-Books is a Jupyter notebook project showcasing the application of deep convolutional neural networks to analyze limit order books. This tool is based on research published in IEEE Transactions on Signal Processing, providing a practical demonstration of the methodologies presented in the paper. It utilizes the publicly available FI-2010 dataset to illustrate how the model architecture is constructed and implemented. The project offers implementations in both TensorFlow (versions 1 and 2) and PyTorch, making it accessible to researchers and developers familiar with either framework. It serves as a valuable resource for understanding and replicating advanced deep learning techniques in financial market analysis.

DLTK (Deep Learning Toolkit) is an open-source Python library designed for medical image analysis, leveraging the TensorFlow framework. It aims to facilitate rapid prototyping of deep learning models and ensure reproducibility in research applications within the medical imaging field. The toolkit provides state-of-the-art methods and models, accelerating research and development. It includes example applications and tutorial notebooks to help users understand its interface with TensorFlow, write custom read functions, and develop their own model functions. DLTK also features a Model Zoo with implementations of current research methodologies.

DriveDreamer is a pioneering world model entirely derived from real-world driving scenarios, specifically designed for autonomous driving research. Unlike other models that focus on gaming or simulated environments, DriveDreamer addresses the critical limitation of lacking real-world representation. It leverages powerful diffusion models to construct comprehensive representations of complex driving environments and employs a two-stage training pipeline. This allows DriveDreamer to first acquire an understanding of structured traffic constraints and then anticipate future states. The tool empowers precise, controllable video generation that faithfully captures real-world traffic scenarios and enables the generation of realistic and reasonable driving policies, opening avenues for interaction and practical applications in autonomous driving.

SwissCognitive is an AI advisory and research firm dedicated to advancing the application of artificial intelligence in business. They offer comprehensive expertise across AI research, advisory services, and ventures, aiming to provide clients with research-based methodologies and industry-driven insights. The firm connects key players within the AI ecosystem, catering to a diverse clientele including organizations, startups, and VC Funds. Their services are designed to help these entities navigate the complexities of AI adoption and strategic implementation, fostering innovation and growth.

DiffBIR is an open-source project providing code and pretrained models for blind image restoration, as presented in the ECCV 2024 paper. It leverages generative diffusion prior to handle various restoration tasks, including blind image super-resolution, blind face restoration (aligned and unaligned), and blind image denoising. The tool offers different model versions, including one trained on the Unsplash dataset with LLaVA-generated captions, and supports features like tiled sampling for large images on low-VRAM GPUs. Users can interact with DiffBIR via a Gradio web interface or through command-line inference scripts, making it accessible for both research and practical applications in image enhancement.

nucleotide-transformer is an open-source repository from InstaDeep AI, dedicated to advancing genomics and transcriptomics through cutting-edge deep learning models. It features a collection of transformer-based genomic language models and innovative downstream applications, including the Nucleotide Transformer (NT), Agro Nucleotide Transformer (AgroNT), SegmentNT, and ChatNT. The platform provides powerful, reproducible, and accessible tools for unlocking new insights from biological sequences, offering pre-trained weights, inference code, and research contributions. It supports various tasks such as functional-track prediction, genome annotation, controllable sequence generation, and single-cell transcriptomics, making it a central hub for AI-driven genomic research.

PDF Summarizer is an AI-powered tool designed to streamline document analysis by summarizing long PDFs. Users can upload documents and engage in multi-file chats, allowing them to ask questions across multiple documents simultaneously, which is ideal for research projects. The system provides detailed or short summaries, extracts key points, and can even create notes, flashcards, and quizzes. A standout feature is its ability to translate any PDF into a preferred language instantly. The tool also offers a side-by-side view, linking questions directly to specific parts of the PDF for easy source checking and deeper exploration without losing context. It supports PDF files up to 50MB and 500 pages, ensuring data security with SOC2 Type II certification.

Emotion-recognition-neural-networks is an open-source project developed for emotion recognition using deep neural networks, specifically with TensorFlow. It employs convolutional neural networks (CNNs) for mood recognition, utilizing the FER-2013 Faces Database which contains 28,709 pictures across 7 emotional expressions. The project provides scripts for data transformation from CSV to NumPy, and supports training models using architectures like AlexNet. While the repository notes that the code might not be actively maintained or fully functional, it serves as a foundational academic project for those interested in exploring DNN-based emotion recognition.

EmotiVoice is a powerful and modern open-source text-to-speech engine available at no cost. It supports both English and Chinese, offering over 2000 distinct voices. A key feature is its emotional synthesis, allowing users to generate speech with a wide range of emotions like happy, excited, sad, and angry. The tool provides an easy-to-use web interface for interactive use and a scripting interface for batch generation. Recent updates include support for tuning voice speed, an app for Mac, an HTTP API with free calls, and voice cloning capabilities. EmotiVoice prioritizes community input and plans to support more languages in the future.

KEATH.ai is an award-winning, intelligent AI marking suite designed for educational assessment. This platform streamlines the grading process, offering rapid evaluation of academic work such as EPQ evaluations, essays, and custom assignments. Beyond just grading, KEATH.ai provides hyper-personalized learning feedback to students, aiming to enhance their educational experience. The tool focuses on delivering unbiased assessment and supporting academic tutoring. It is built to assist educators in efficiently managing their assessment workload while ensuring students receive tailored insights for improvement.

EnCodec is a state-of-the-art deep learning-based audio codec developed by Facebook Research. It offers high-fidelity neural audio compression for both mono 24 kHz audio and stereo 48 kHz audio. The tool provides two multi-bandwidth models: a causal model for 24 kHz monophonic audio and a non-causal model for 48 kHz stereophonic audio, trained on music-only data. Users can compress audio to various bitrates, ranging from 1.5 kbps to 24 kbps, depending on the model. EnCodec also includes pre-trained language models for further compression without quality loss and can be integrated with Hugging Face Transformers for scalable use. It supports direct command-line usage for compression, decompression, and extracting discrete audio representations.

mPLUG-Owl is a family of multi-modal large language models (MLLMs) designed to enhance language models with multimodality through a modular approach. The project includes several iterations: mPLUG-Owl, mPLUG-Owl2, and mPLUG-Owl3, each building upon the previous version to offer improved capabilities. mPLUG-Owl2, for instance, was accepted by CVPR 2024 as a Highlight, and mPLUG-Owl2.1 provides a Chinese-enhanced version. The latest iteration, mPLUG-Owl3, focuses on long image-sequence understanding. The source code and weights for these models are available on HuggingFace, making them accessible for researchers and developers to integrate and experiment with.

mteb (Massive Text Embedding Benchmark) is an open-source Python library designed for comprehensive evaluation of text and multimodal embeddings. It offers a standardized framework to benchmark the performance of different embedding models across a wide array of tasks, including classification, clustering, semantic textual similarity (STS), retrieval, and reranking. The tool supports both monolingual and multilingual evaluations, with a focus on reproducibility and ease of use. Developers and researchers can use mteb to select models, define custom models, run evaluations, and analyze results, contributing to an interactive leaderboard that tracks the state-of-the-art in embedding performance. Its modular design allows for easy integration of new models, datasets, and benchmarks.

The Hugging Face Evaluation Guidebook is a comprehensive resource for understanding and implementing Large Language Model (LLM) evaluation. It provides both practical insights and theoretical knowledge, drawing from the experience of managing the Open LLM Leaderboard and designing the lighteval framework. The guidebook covers various evaluation methods, including automatic benchmarks, human evaluation, and LLM-as-a-judge approaches. It offers guidance on designing custom evaluations, troubleshooting common issues, and provides tips and tricks for both beginner and advanced users. Additionally, it includes sections on general LLM knowledge, such as model inference and tokenization, making it a valuable resource for anyone looking to ensure their LLM performs effectively.

DyNet is a powerful open-source neural network library, primarily developed by Carnegie Mellon University, with contributions from many others. Written in C++ and offering Python bindings, it's engineered for efficiency on both CPU and GPU architectures. A key differentiator is its ability to handle dynamic neural network structures, which can adapt and change for each training instance. This makes DyNet particularly well-suited for complex natural language processing tasks, where it has been successfully applied to build state-of-the-art systems for syntactic parsing, machine translation, and morphological inflection. The toolkit provides comprehensive documentation, tutorials for both C++ and Python, and examples to help users get started with its auto-batching feature and other functionalities.

DropoutUncertaintyExps is an open-source project containing the experimental code for the paper "Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning." The repository provides a framework for researchers to replicate and build upon the uncertainty experiments, with adaptations reflecting community feedback and bug fixes. It is based on José Miguel Hernández-Lobato's work on probabilistic backpropagation for scalable learning of Bayesian Neural Networks. The code utilizes datasets from the UCI machine learning repository, with specific data splits to ensure comparability of results. It details the methodology for hyperparameter tuning using grid-search and reports RMSE and log-likelihood metrics for various datasets, offering a valuable resource for academic research in deep learning uncertainty.

The PatchCamelyon (PCam) benchmark is a challenging image classification dataset designed for deep learning in medical imaging. It comprises 327,680 color images (96 x 96px) extracted from histopathologic scans of lymph node sections. Each image is annotated with a binary label indicating the presence of metastatic tissue, making it ideal for training and evaluating machine learning models for metastasis detection. PCam is larger than CIFAR10 but smaller than ImageNet, allowing models to be trained on a single GPU within a few hours. It serves as a valuable resource for fundamental machine learning research on topics such as active learning, model uncertainty, and explainability, particularly within the medical domain. The dataset is provided in gzipped HDF5 files and includes training, validation, and test sets with balanced positive and negative examples.

Paper2Any is an AI-powered tool designed to streamline the creation of academic and technical visual content from research papers, text, or topics. It excels in multimodal workflows, allowing users to generate editable research figures, technical route diagrams, experimental plots, and presentation slides with a single click. Key capabilities include Paper2Figure for scientific diagrams, Paper2Diagram/Image2Drawio for editable diagrams, and Paper2PPT for creating slide decks. The tool also offers specialized features like Paper2Rebuttal for drafting responses, PDF2PPT for layout-preserving conversions, and Image2PPT for turning images into structured slides. With features like an Image Model Playground, smart beautification (PPTPolish), and a Knowledge Base for semantic search, Paper2Any provides a comprehensive solution for researchers and academics to visualize and present their work efficiently.

open-llms is a comprehensive GitHub repository that serves as a curated list of open Large Language Models (LLMs) explicitly licensed for commercial use, including Apache 2.0, MIT, and OpenRAIL-M. This resource is invaluable for developers, researchers, and businesses looking to integrate open-source LLMs into their applications without licensing concerns. The repository details each model's release date, available checkpoints, associated research papers or blog posts, parameter sizes, context lengths, and specific licenses. It also includes a dedicated section for open LLMs tailored for code generation, offering insights into models like SantaCoder, CodeGen2, and StarCoder. Contributions to the list are welcomed, ensuring it remains up-to-date with the latest commercially viable open LLM releases.

Pubcompare is an AI-powered platform designed for researchers to find, compare, and evaluate experimental protocols. It leverages AI to dissect and index over 40 million protocols from peer-reviewed publications, preprints, and patents. The tool extracts specific parameters like concentrations, incubation times, and cell counts, providing statistically consolidated data without interpretation. Users can generate consensus reports, compare protocols side-by-side, and identify the most relevant and cited ones to assess reproducibility. Pubcompare is primarily designed for Life sciences and chemistry but is adaptable for any field requiring detailed experimental protocols.

Open Researcher is a powerful AI-powered research tool designed to streamline the process of searching, analyzing, and understanding web content. It leverages Firecrawl's web scraping capabilities to gather accurate and up-to-date information, which is then processed by advanced AI reasoning, powered by Anthropic's Claude. Key features include an AI-powered search, a real-time thinking display that shows the AI's reasoning process, smart citations for automatic source tracking, and a split-view interface for side-by-side chat and search results. This tool is ideal for anyone needing to efficiently research and synthesize information from the web, providing a transparent and well-sourced analysis.