Research & Education

MiniGPT4-video offers official code for the Goldfish model, designed for understanding arbitrarily long videos, and MiniGPT4-video itself, tailored for short video understanding. This tool advances multimodal Large Language Models (LLMs) by integrating visual and textual tokens for comprehensive video analysis. Goldfish addresses challenges in long video processing through an efficient retrieval mechanism that identifies relevant video clips, making it suitable for applications like movies or TV series. MiniGPT4-video generates detailed descriptions for video clips, facilitating the retrieval process for Goldfish. The project also introduces the TVQA-long benchmark for evaluating long video comprehension and demonstrates significant performance improvements over existing state-of-the-art methods in both long and short video understanding.

ml-cvnets is a comprehensive computer vision toolkit developed by Apple, designed for researchers and engineers to efficiently train a wide array of computer vision models. It supports both standard and novel mobile- and non-mobile architectures for tasks such as object classification, object detection, semantic segmentation, and foundation models like CLIP. The library is built on Python 3.10+ and PyTorch, offering features like automatic data augmentation (RangeAugment, AutoAugment, RandAugment) and enhanced distillation support. It includes a model zoo with various CNNs (MobileNet, EfficientNet, ResNet) and Transformers (Vision Transformer, MobileViT, SwinTransformer), making it a versatile platform for advanced computer vision research and development.

AI Singapore is a national program launched in May 2017, dedicated to fostering advanced AI capabilities within Singapore. It serves as a nexus for Singapore-based research institutions, AI startups, and established companies, facilitating collaborative efforts in use-inspired research, knowledge creation, tool development, and talent cultivation. The initiative focuses on key areas such as AI Research, Governance, Technology, Innovation, and Products, aiming to generate significant social and economic impact. It also offers various talent development programs, including the AI Apprenticeship Programme (AIAP) and LearnAI, to equip professionals and students with essential AI skills.

apic.ai is a leading specialist in automated pollinator monitoring, leveraging artificial intelligence and edge computing to provide reliable and fully automated behavioral assessments of bees and bumblebees. Their minimal-invasive camera system, installed at hive entrances, visually detects all movement in and out of the colony. The collected video footage is analyzed using AI algorithms, providing real-time data on activity, foraging behavior, pollen diversity, mortality, and individual size. This technology helps manufacturers and testers of plant protection products improve risk assessment, enables seed producers to develop practices that enhance crop pollination, and supports companies in designing pollinator-friendly habitats. The scientific approach ensures validated methods and verifiable results, making even subtle effects of substances and environmental factors visible.

awesome-neural-geometry is a comprehensive, curated collection of resources and research focused on the geometry of representations within the brain, deep neural networks, and related fields. This open-source repository, collaboratively generated on the Symmetry and Geometry in Neural Representations Slack Workspace, includes educational materials like textbooks, notes, courses, and videos covering topics such as Abstract Algebra, Differential Geometry, Information Geometry, Dynamics, Topology, and Geometric Machine Learning. It also lists computational neuroscience resources, datasets, software libraries like Geomstats and E3NN, and relevant conferences and workshops. The project is a work-in-progress and actively encourages contributions via pull requests.

Courtroom5 is an award-winning legal platform designed to empower self-represented litigants in U.S. state and federal civil courts. It offers the LAW Accelerator™, a comprehensive program that provides case management, guidance, and custom document creation. The platform utilizes AI-powered tools to explain legal documents, analyze case facts, research relevant law, build arguments, and generate court filings. Users receive procedural guidance at every stage, enabling them to make informed strategic decisions. Courtroom5 also fosters a supportive community with workshops, office hours, and peer support, helping members succeed without a lawyer.

Playground is an open-source platform dedicated to AI research in multi-agent learning, primarily through the game Pommerman, a clone of Bomberman. Researchers and AI enthusiasts can submit agents they have trained to compete in regular competitions across three variants: Free For All (FFA), Team (2v2 with partial observability), and Team Radio (2v2 with limited communication). The platform aims to provide approachable benchmarks for multi-agent learning, foster contributions to multi-agent and communication research, and offer a competitive environment for AI development. It supports training agents with popular libraries like TensorForce and provides an example training script. Submissions are handled via Docker containers, ensuring agent safety and fair play.

pytorch-pruning is an open-source PyTorch implementation of the paper "Pruning Convolutional Neural Networks for Resource Efficient Inference." This tool is designed to optimize deep learning models by reducing their size and improving inference speed. It achieves this by systematically removing filters from convolutional layers. The project demonstrates its effectiveness by pruning a VGG16-based classifier on a small dog/cat dataset, resulting in a significant 3x reduction in CPU runtime and a 4x reduction in model size. While currently pruning filters sequentially, the project notes that future improvements could include a single-pass pruning mechanism for greater efficiency. It also aims to support additional architectures beyond VGG, such as VGG with batch normalization.

prm800k is an open-source dataset and accompanying tools, released by OpenAI, that provides 800,000 step-level correctness labels for large language model (LLM) solutions to mathematical problems from the MATH dataset. This resource is crucial for researchers and developers aiming to enhance the mathematical reasoning capabilities of AI models through process supervision. The repository includes raw labels, instructions for labelers, Python grading logic for answer correctness, and non-standard MATH train/test splits. It also contains scored samples used to evaluate large-scale ORM and PRM models, making it a comprehensive resource for advancing AI in mathematics.

SAMv2 Mask Generator is an AI-powered tool available as a Hugging Face Space by lightly-ai, designed for image segmentation tasks. Users can upload any image and interactively define objects of interest by drawing bounding boxes around them. The tool then automatically generates precise segmentation masks, highlighting the selected objects within the image. This functionality is particularly useful for various computer vision applications, including object detection, image analysis, and data labeling, providing a straightforward method to isolate and analyze specific elements within visual data. It offers a practical solution for researchers, developers, and data annotators working with image datasets.

The Center for Digital Health and Artificial Intelligence at Johns Hopkins (CDHAI) is a research initiative focused on exploring the convergence of digital technologies, artificial intelligence, and healthcare. CDHAI's primary objective is to conduct rigorous academic research that generates actionable insights into how these technological innovations can effectively address critical challenges within the healthcare sector. This includes improving healthcare quality, empowering patients through digital tools, and advancing health equity. The center's work contributes to the broader understanding of AI's potential and implications in medical and public health contexts, fostering advancements that benefit both practitioners and patients.

PyABSA is a modular and reproducible open-source framework designed for Aspect-based Sentiment Analysis (ABSA), bridging the gap from research to production. It offers a unified API for training, evaluation, and inference across multiple ABSA subtasks, including Aspect Polarity Classification (APC), Aspect Term Extraction & Polarity Classification (ATEPC), Aspect Sentiment Triplet Extraction (ASTE), and Aspect Category Opinion Sentiment Triplet Extraction (ASQP/ACOS). The framework comes with a Model Zoo of available checkpoints that auto-download, visualization tools for evaluation metrics, and helpers for dataset annotation. Additionally, PyABSA supports text augmentation for classification and adversarial defense, along with automatic device selection for CPU/GPU. It is ideal for researchers and developers working with sentiment analysis and natural language processing tasks.

recommendation is an Open Source workshop resource designed for individuals interested in building recommendation systems using both machine learning and deep learning. The resource delves into the theoretical underpinnings, including ML & DL formulation, prediction vs. ranking, and similarity metrics. It explores different paradigms such such as content-based, collaborative filtering, knowledge-based, hybrid, and ensemble approaches. Users can learn to work with various data types, including tabular, images, and text, and implement models like Matrix Factorization, Auto-Encoders, Wide & Deep, and Sequence Modelling. The workshop also covers practical aspects like setup, encoding, design, training, and evaluation, providing a comprehensive guide for developing and deploying recommendation systems.

ReID-Survey is an open-source GitHub repository dedicated to deep learning for person re-identification. It offers comprehensive surveys, including an in-depth analysis of Transformer's impact across various Re-ID directions and a survey on deep learning for person re-identification with a powerful AGW baseline. The repository provides implementations for unsupervised Re-ID, cross-modality visible-infrared unsupervised Re-ID, and a unified experimental standard for animal Re-ID. Researchers can find code, datasets, and detailed experimental results for various Re-ID tasks, making it a valuable resource for advancing research in this field.

Quantitative-Research-Projects is a curated GitHub repository offering a collection of quantitative finance research projects. The projects delve into various aspects of financial analysis, including sector rotation, multi-factor models, and advanced AI-driven strategies utilizing machine learning and deep learning techniques. These strategies are applied across high, mid, and low frequencies, providing a comprehensive view of quantitative finance. Each project within the repository is accompanied by full code and detailed analysis, making it a valuable resource for researchers and practitioners. The collection is continuously updated, ensuring access to the latest research and methodologies in the field.

robustlearn is an open-source library developed by Microsoft for research in robust machine learning, focusing on responsible AI. It offers a unified platform for exploring various aspects of robustness, including adversarial and backdoor attack and defense mechanisms, out-of-distribution (OOD) generalization, and safe transfer learning. The library hosts several projects like SpecFormer for adversarial robustness in Vision Transformers, NMtune for understanding label noise in pre-training, and RiFT for improving generalization of adversarial training. It also includes projects addressing OOD generalization for time series classification, domain-specific risk minimization, and activity recognition. robustlearn is designed to be extensible, allowing researchers to develop and test their own robust machine learning models.

SPO (Self-Supervised Prompt Optimization) is an AI tool hosted on Hugging Face Spaces designed to enhance the performance of language models by optimizing user prompts. It allows users to create or select templates, configure various settings, and initiate an optimization process to achieve better responses from AI models. This application is particularly useful for prompt engineers and researchers looking to fine-tune their interactions with large language models, ensuring more accurate and relevant outputs through a self-supervised learning approach. The tool aims to streamline the prompt engineering workflow, making it easier to experiment with and improve prompt effectiveness.

StyleGAN-Human Interpolation is a web-based tool hosted on Hugging Face Spaces, designed for generating and manipulating human faces using AI. It leverages StyleGAN models to create realistic synthetic faces, offering users the ability to explore the capabilities of this advanced generative adversarial network. The primary function of the tool is to produce a series of images that smoothly transition between two distinct, randomly generated human images. Users can control this interpolation process by adjusting parameters such as seed values and truncation psi, which influence the randomness and realism of the generated faces. This makes it a valuable resource for researchers, artists, and enthusiasts interested in AI-driven image synthesis and the nuances of facial generation.

SWE-RL is an official codebase for "Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution," designed to scale reinforcement learning-based LLM reasoning for real-world software engineering tasks. It leverages open-source software evolution data and rule-based rewards to improve LLM performance. The codebase includes prompt templates and a flexible reward function API that supports various editing formats, including sequence similarity for search/replace changes and unified diffs. Additionally, SWE-RL features an Agentless Mini component for fast asynchronous inference, code refactoring, file-level localization, and repair, supporting OpenAI-compatible endpoints and Hugging Face models like Llama-3.3-70B-Instruct.

sqlite-vss is a SQLite extension designed to bring vector search capabilities directly into SQLite databases, leveraging the Faiss library for efficiency. It enables developers to build semantic search engines, recommendation systems, and question-and-answering tools by storing and querying vector embeddings. While not actively developed, with efforts now focused on sqlite-vec, it offers a robust solution for integrating vector search into applications using SQLite. Users can create virtual tables to store high-dimensional embeddings and perform k-nearest neighbor searches. It supports various languages through bindings like Python, Node.js, Deno, Ruby, Elixir, Go, and Rust, making it accessible to a wide range of developers.

The synthetic-personality-dataset offers a high-fidelity collection of 10,000 synthetic records designed to simulate the behavioral and social patterns of introverted and extroverted individuals. Generated using Syncora.ai's synthetic data engine, this dataset ensures zero privacy risk while preserving real-world behavioral distributions. It is ideal for researchers, data scientists, and AI developers focused on personality prediction, behavioral modeling, machine learning experiments, and social science research. The dataset includes features like time spent alone, social event attendance, social media posting habits, and a personality target label, making it suitable for various analytical and ML use cases without compromising privacy or ethical concerns.

TimeCapsuleLLM is an innovative open-source project focused on creating language models (LLMs) trained exclusively on data from specific historical periods and geographic locations. The primary goal is to mitigate modern biases inherent in contemporary LLMs and accurately emulate the linguistic style, vocabulary, and worldview of a chosen era. The project has developed several versions, including v0, v0.5, v1, and v2, with increasing dataset sizes and model parameters, built on architectures like nanoGPT, Phi 1.5, and llamaforcausallm. It emphasizes Selective Temporal Training (STT) where all training data is curated from a defined historical window, ensuring the model's knowledge and language reflect that period without modern influence. The project provides core training scripts, tokenizer building tools, and detailed documentation for researchers and developers interested in historical language modeling.

talk2arxiv is an open-source Retrieval-Augmented Generation (RAG) system specifically designed for academic paper PDFs. It enables users to chat with any ArXiv paper by simply modifying the paper's URL. The system features PDF parsing using GROBID for efficient text extraction, a custom chunking algorithm that organizes text by logical sections and recursive subdivision, and Cohere's EmbedV3 model for accurate text embeddings. It integrates with Qdrant for vector database storage and querying, which also caches research papers to avoid re-embedding. A reranking process ensures contextual relevance based on user input. The frontend is built with Typescript, ReactJS, TailwindCSS, and NextJS, while the backend utilizes Flask, Gunicorn, and Nginx.

Texar is a comprehensive toolkit designed to support a broad range of machine learning tasks, with a particular focus on natural language processing and text generation. Built on TensorFlow, it offers a rich library of modular and easy-to-use ML components and functionalities, enabling both researchers and practitioners to rapidly prototype and experiment with models. Key features include support for pre-trained models like BERT, GPT2, and XLNet, and full customizability at multiple abstraction levels. Texar is versatile, supporting various tasks, models, algorithms, data processing, and evaluation methods, from encoder-decoder architectures to reinforcement learning and adversarial learning. It emphasizes modularity for maximum re-use and clean APIs, based on a principled decomposition of learning, inference, and model architecture. The toolkit also supports distributed model training with multiple GPUs and provides extensive documentation and examples.