Research & Education

Calcish is a versatile macOS application that integrates a programmable calculator, an enhanced JavaScript engine, AI capabilities, and an intuitive notebook into a single tool. It allows users to instantly solve complex math problems, work with big numbers, fractions, vectors, and matrices, and perform advanced scientific computing. The enhanced JavaScript engine, powered by QuickJS, supports arbitrary-precision integers and decimals, going beyond typical browser console limitations. Users can organize calculations, code, and AI chats in editable notebooks, with rich output and easy export options. AI integration supports both local and remote models for brainstorming, code generation, and data analysis, all accessible via a global hotkey for quick access.

MARLlib is a comprehensive, open-source library designed for Multi-agent Reinforcement Learning (MARL), leveraging Ray and its RLlib toolkit. It offers a unified platform for researchers and developers to create, train, and evaluate MARL algorithms across a wide array of tasks and environments. Key features include support for all task modes (cooperative, collaborative, competitive, mixed), a Gym-like interface for multi-agent environments, and flexible parameter-sharing strategies. MARLlib provides 18 pre-built algorithms with an intuitive API, making it accessible even for those new to MARL. Users can customize model architectures, policy sharing, and access over a thousand released experiments. It is compatible with Linux operating systems and offers step-by-step installation or Docker-based usage.

Yoodli AI is an enterprise AI roleplay platform designed to enhance communication skills through interactive simulations. It offers a private, judgment-free environment for users to practice pitches, demos, crucial conversations, and public speaking. The platform provides real-time feedback on content, delivery, and progress over time, utilizing AI-powered follow-up questions. Yoodli AI is trusted by major companies like Google and Sandler for sales enablement, partner training, and learning & development. It supports multi-persona roleplays to simulate group presentations or interview panels, and integrates with existing ecosystems for automatic roleplay assignment, progress tracking, and data synchronization. The tool is SOC 2 Type 2 certified and GDPR compliant, ensuring data security and privacy.

cnn-text-classification-pytorch is an open-source implementation of Convolutional Neural Networks (CNNs) for sentence classification, built using PyTorch. This tool is based on the model described in Kim's influential paper on CNNs for Sentence Classification. It offers a practical framework for developers to perform text classification tasks, providing consistent results with the original research. The implementation has been updated to be compatible with modern PyTorch versions (2.0+), removing deprecated dependencies like `torchtext` and fixing various runtime errors. It supports datasets like MR and SST, includes options for different optimizers (Adam, Adadelta), and allows for easy training, testing, and prediction of text sentiment.

MM-EUREKA is a cutting-edge project exploring the frontiers of multimodal reasoning through rule-based reinforcement learning. It introduces powerful models such as MM-Eureka-Qwen-7B and MM-Eureka-Qwen-32B, which significantly advance performance in multidisciplinary K12 and mathematical reasoning tasks. The project has iterated on model architecture, algorithms, and data, moving from InternVL to the more robust Qwen2.5-VL base models. Key improvements include enhanced online filtering, adaptive online rollout adjustment (ADORA), and novel RL algorithms like Clipped Policy Gradient Optimization with Policy Drift (CPGD). MM-EUREKA also open-sources a comprehensive pipeline, including self-collected MMK12 datasets, to foster further research and development in multimodal AI.

DeePMD-kit is a Python/C++ package designed to facilitate the creation of deep learning-based models for interatomic potential energy and force fields, and to perform molecular dynamics simulations. It addresses the accuracy-versus-efficiency dilemma in molecular simulations by leveraging deep learning. The package is highly modularized and interfaces with popular deep learning frameworks like TensorFlow, PyTorch, JAX, and Paddle, as well as high-performance classical and quantum MD packages such as LAMMPS, i-PI, and GROMACS. It implements the Deep Potential series models, which have been successfully applied to various systems, including organic molecules, metals, and semiconductors. DeePMD-kit also supports MPI and GPU for efficient parallel and distributed computing, making it suitable for complex scientific research.

Deepgaze is an open-source computer vision library designed for human-computer interaction, providing advanced capabilities for analyzing human behavior through visual data. It leverages Convolutional Neural Networks (CNNs) for precise head pose and gaze direction estimation, which is crucial for understanding a person's focus of attention, even when eyes are obscured or far from the camera. Beyond CNN-based estimation, Deepgaze incorporates features like skin detection via backprojection, robust motion detection and tracking, and saliency map generation using the FASA algorithm. Built on OpenCV and TensorFlow, it offers optimized, state-of-the-art algorithms, making complex implementations accessible with just a few lines of code for both beginners and advanced users in computer vision and machine learning.

Mindgrasp AI is an advanced AI-powered learning platform designed to enhance study habits and academic success for students, professionals, and self-learners. It processes various content types, including PDFs, DOCX, MP3, MP4, Powerpoints, online articles, and YouTube/Vimeo links, instantly converting them into comprehensive study materials. Key features include AI-generated notes, summaries, flashcards, and quizzes, all built on cognitive science principles to promote faster learning and better retention. The platform also offers a 24/7 AI tutor for immediate concept clarification and personalized learning support. Mindgrasp AI tracks learning progress for each study session, allowing users to monitor their coverage and remaining review tasks. It supports learning across multiple devices and in over 20 languages, making it a versatile tool for diverse educational needs.

ms-swift is a comprehensive, open-source framework developed by the ModelScope community, designed for fine-tuning and deploying large language models (LLMs) and multimodal large models (MLLMs). It supports over 600 text-only LLMs and 400 MLLMs, offering full-pipeline capabilities from training to inference, evaluation, quantization, and deployment. The framework integrates advanced training technologies, including Megatron parallelism (TP, PP, CP, EP) for acceleration and a rich family of GRPO reinforcement learning algorithms. ms-swift also supports various fine-tuning methods like LoRA, QLoRA, and DoRA, and provides memory optimization techniques such as Flash-Attention 2/3. It offers a Web-UI interface for simplified training, inference, evaluation, and quantization workflows, making it accessible for a wide range of users.

LimSim is a Long-term Interactive Multi-scenario traffic Simulator designed to provide continuous simulation capabilities within complex urban road networks. It features long-term traffic flow generation based on demand and route planning, diverse behavioral models for heterogeneous driving styles, and interactivity to manage sophisticated vehicle interactions. The simulator supports multiple scenarios including freeways, signalized intersections, roundabouts, and overpasses. LimSim also includes a cross-platform GUI for visualizing simulations, road networks, and ego-vehicle status. It can generate log reports and extract key scenarios for evaluation after long-term simulations. Notably, LimSim supports co-simulation with CARLA and SUMO, ensuring identical vehicle status across platforms, and LimSim++ offers Multimodal LLM support.

LMM-R1 is an open-source project designed to enhance the reasoning capabilities of 3B Large Multimodal Models (LMMs) by extending the OpenRLHF framework. It addresses the challenges of limited parameter capacity and scarce high-quality multimodal reasoning data through a novel two-stage rule-based RL approach. The first stage, Foundational Reasoning Enhancement (FRE), builds strong reasoning foundations using text-only data. The second stage, Multimodal Generalization Training (MGT), extends these capabilities to multimodal domains. LMM-R1 supports various LMMs like Qwen2.5-VL, Phi3.5-V, and Phi4-Multimodal, and offers distributed PPO and REINFORCE++/RLOO implementations based on Ray, achieving significant speedups. It also integrates with vLLM for accelerated generation, FlashAttention2, and supports QLoRA/LoRA for efficient fine-tuning.

MLSys-NYU-2022 is an open-source repository containing slides, scripts, and materials from the Machine Learning in Finance course at NYU Tandon, 2022. Developed by Prof. Ethan Rosenthal and Jacopo Tagliabue, this resource focuses on practical applications of ML in finance, emphasizing industry-standard tools and good coding habits. It covers topics like MLOps, fraud detection, recommender systems, and evaluation metrics. The course materials are structured by week, with self-contained folders including READMEs, scripts, notebooks, and slides, each with a dedicated `requirements.txt` for reproducibility. It also introduces tools like Metaflow, Streamlit, Comet, and Flask for building and deploying ML pipelines and applications.

mml-book.github.io serves as the official companion webpage for the book "Mathematics For Machine Learning" by Marc Peter Deisenroth, A Aldo Faisal, and Cheng Soon Ong. This resource is designed to motivate and equip individuals with the necessary mathematical skills to understand advanced machine learning techniques. The site provides supplementary materials, including exercises for the mathematical foundations section and Jupyter notebooks for the example machine learning algorithms. These notebooks can be run live on Google Colab, offering an interactive learning experience. The book aims to be concise, focusing on core mathematical concepts rather than exhaustive coverage of advanced ML techniques, making this companion site a valuable educational aid.

OpenML is a collaborative online machine learning platform designed to facilitate the sharing and organization of data, machine learning algorithms, and experimental results. It aims to create a frictionless, networked ecosystem where scientists and practitioners can easily integrate their existing processes and tools to collaborate globally. The platform provides significant benefits for science by enabling rapid building upon others' results, answering complex questions quickly through prior experiments, and making larger studies feasible. For scientists, it saves time on routine duties, compares new experiments to the state of the art, and offers potential for new discoveries and publications. OpenML also serves as a valuable learning environment for students and citizen scientists, allowing them to explore state-of-the-art methods and contribute their own work.

Osprey is a cutting-edge computer vision tool that enhances multimodal large language models (MLLMs) by incorporating pixel-wise mask regions into language instructions. This innovative approach enables fine-grained visual understanding, allowing Osprey to generate detailed semantic descriptions, including both short and elaborate explanations, based on specific input mask regions. It seamlessly integrates with Segment Anything Model (SAM) in various modes like point-prompt, box-prompt, and segmentation everything, to extract and describe semantics associated with particular parts or objects within an image. Osprey is built upon the LLaVA-v1.5 codebase and is designed for researchers and developers working on advanced visual instruction tuning and pixel-level image analysis.

PyTorch-BayesianCNN provides an implementation of Bayesian Convolutional Neural Networks (CNNs) with variational inference, specifically utilizing Bayes by Backprop, within the PyTorch framework. This tool allows researchers and developers to build CNNs that can infer intractable posterior probability distributions over weights, offering a significant advantage over traditional frequentist approaches by providing uncertainty estimations. It includes two types of Bayesian layer implementations: BBB (Bayes by Backprop) and BBB_LRT (Bayes by Backprop with Local Reparametrization Trick), which enhances sampling efficiency. The repository supports standard datasets like MNIST, CIFAR10, and CIFAR100, and includes implementations of common models such as AlexNet and LeNet, making it a valuable resource for experimenting with Bayesian deep learning and understanding model uncertainty.

pytorch_active_learning is an open-source PyTorch library designed for active learning, accompanying the "Human-in-the-Loop Machine Learning" book. It offers a range of active learning methods, including Least Confidence, Margin of Confidence, Ratio of Confidence, and Entropy sampling. The library also supports more advanced techniques like Model-based Outlier sampling, Cluster-based sampling, and various forms of Active Transfer Learning. It is suitable for researchers and practitioners looking to experiment with and apply active learning strategies in computer vision and natural language processing, with a focus on real-world diversity to avoid bias. The code is stand-alone and can be easily integrated with existing PyTorch installations.

Outline AI is an AI-powered tool designed to simplify the outline creation process. Users can generate comprehensive outlines by simply inputting their desired content or by providing source material such as websites, PDFs, images, and audio files. The tool leverages the latest AI technology to summarize information and structure it into a clear, organized outline. It is highly beneficial for brainstorming, academic writing, research structuring, preparing presentations, and organizing notes, offering a streamlined approach to content organization and idea development. The platform aims to enhance productivity by automating the initial structuring phase of various projects.

RoseTTAFold is a deep learning model and script package designed for the accurate prediction of protein structures and interactions. This tool is an official implementation of the RoseTTAFold architecture, which employs a 3-track neural network to achieve its predictions. It is primarily intended for research in computational biology, enabling scientists to model complex protein structures and protein-protein interactions (PPIs). The package includes scripts for installation, dependency management, and running predictions for both monomer structures and complex modeling. It also features a faster 2-track version for PPI screening, making it a versatile tool for advanced biological research.

Ditto Transcript Generator is a privacy-first browser extension designed to extract video transcripts from various online platforms. It operates entirely locally within your browser, ensuring no data is sent to external servers, making it a secure choice for users concerned about privacy. The tool supports major platforms including YouTube, Vimeo, Rumble, Udemy, Coursera, Teachable, and Kajabi. It converts video content into clean, paragraph-formatted text by intelligently stripping timestamps and reconstructing sentences for improved readability. Ditto is open-source, free to use, and requires no login or account, offering unlimited transcript extractions without paywalls.

SimGNN is a PyTorch implementation of a novel neural network approach designed for fast graph similarity computation, as detailed in the WSDM 2019 paper. It addresses the computational burden of traditional methods like Graph Edit Distance (GED) and Maximum Common Subgraph (MCS) while maintaining high performance. The tool employs a learnable embedding function to map graphs into embedding vectors, providing a global summary. A key feature is its attention mechanism, which emphasizes important nodes for specific similarity metrics. Additionally, SimGNN includes a pairwise node comparison method to supplement graph-level embeddings with fine-grained node-level information. This approach leads to better generalization on unseen graphs and offers quadratic time complexity in the worst case. Experimental results demonstrate its effectiveness and efficiency, achieving smaller error rates and significant time reductions compared to existing baselines.

Self-Driving-Car-in-Video-Games is an open-source project featuring a supervised deep neural network designed to learn autonomous driving within video games, specifically Grand Theft Auto V. The model, named T.E.D.D. 1104, is trained using extensive human-labeled data, recording gameplay and key inputs to teach it how to navigate various vehicles under different weather conditions. It approaches the task as a classification problem, taking a sequence of five images as input and predicting the correct keyboard or Xbox controller inputs. The project provides pretrained models of varying sizes (XXL, M, S) and includes all necessary files for data generation, training, and real-time inference, primarily supporting Windows 10/11 for gameplay interaction.

Iridium is an AI-powered fitness coach that revolutionizes strength training and nutrition tracking. It moves beyond simple logbooks by analyzing your previous workout volume, current recovery status, and muscle balance to generate optimal, adaptive workout sessions. The app intelligently builds workouts based on available equipment and time, incorporating superset logic and ensuring only recovered muscle groups are targeted. Beyond workouts, Iridium offers smart macro logging with barcode scanning, AI description, and photo recognition for precise nutrition tracking. It also provides detailed recovery data, including HRV, resting heart rate, sleep, and muscle-specific recovery percentages, ensuring users train smarter and avoid overtraining. With an Apple Watch app, video guides for exercises, and progress tracking, Iridium offers a comprehensive solution for personalized fitness.

TensorFlow Federated (TFF) is an open-source framework designed for machine learning and other computations on decentralized data. It specifically supports Federated Learning (FL), an approach where a shared global model is trained across many participating clients while their sensitive training data remains local. This framework enables developers to utilize included federated learning algorithms with their existing TensorFlow models and data, or to experiment with novel algorithms. TFF provides both a high-level Federated Learning (FL) API for applying federated training and evaluation, and a lower-level Federated Core (FC) API for expressing new federated algorithms. It includes a single-machine simulation runtime for experiments, making it suitable for researchers and developers exploring privacy-preserving machine learning.