Research & Education

The 'State of Open Source AI' is a comprehensive guide presented as an ebook, designed to bring clarity to the rapidly evolving landscape of open-source AI. It covers a wide range of topics, from model evaluations to deployment strategies, serving as a valuable resource for anyone looking to understand current innovations and avoid FOMO in the fast-paced AI world. The project is hosted on GitHub, encouraging community contributions to keep the content up-to-date. It also provides resources for discussion, including a dedicated Discord channel, Twitter, and a newsletter, fostering engagement within the open-source AI community.

Structure_knowledge_distillation is an open-source repository providing the official code for the paper 'Structured Knowledge Distillation for Semantic Segmentation' (CVPR 2019 ORAL) and its extension to other dense prediction tasks. This tool facilitates the transfer of structured knowledge from a larger, more complex teacher model to a smaller, more efficient student model. It includes implementations for pixel-wise, pair-wise, and holistic distillation methods, demonstrating improved performance on tasks like semantic segmentation, object detection, and depth estimation. The repository offers pre-trained models and detailed instructions for compiling and running tests, making it a valuable resource for researchers and practitioners in the field of computer vision.

tensorflow-deepq offers a foundational demonstration of deep Q-learning principles implemented with Google TensorFlow. This open-source project provides a basic framework for understanding and experimenting with reinforcement learning tasks, particularly focusing on how a DeepQ controller can learn strategies within a simulated environment. Users can define custom controllers and simulations, observe states, collect rewards, and perform actions. While the repository is noted as obsolete by its maintainer, with a more complete implementation available from OpenAI, it still serves as a valuable educational resource for those looking to grasp the core concepts of deep Q-learning and its application using TensorFlow. It includes functionalities for creating GIF animations of learned strategies.

Probable offers real-time probability tracking for 16 major global events across various domains including geopolitics, markets, climate, health, energy, and cybersecurity. The platform provides three smart predictions per topic, which are updated live every 5 seconds, ensuring users have access to the most current information. It leverages AI to offer insights and predictions, making it a valuable resource for monitoring world signals and potential future developments. Probable is designed to be free forever, making it accessible to a wide audience interested in tracking global trends and risks.

Total-Text-Dataset is a comprehensive, word-level based English curve text dataset designed to facilitate research in text detection and recognition. It comprises 1555 images featuring more than three different text orientations: horizontal, multi-oriented, and curved, making it unique among existing datasets. The dataset is regularly updated with detection and recognition leaderboards, showcasing the performance of various methods. It also provides an updated guided annotation toolbox for scene text image annotation and includes pixel-level and text-level ground truth data. Researchers can leverage this dataset for training and benchmarking models, particularly for arbitrary-shaped text reading tasks, and it has been extended into the larger ArT dataset.

Vision Papers is a Hugging Face Space designed to help users conveniently explore summaries of vision papers. This tool allows researchers and students to quickly grasp the key points of academic research in the field of computer vision and vision language models. By browsing through the left tab, users can discover more resources and stay up-to-date with the latest advancements. The platform aims to make complex research papers more accessible, saving time and effort for those looking to understand cutting-edge AI developments.

LOTUS Depth is an AI tool designed for advanced depth estimation, allowing users to generate detailed depth maps from uploaded images or videos. The application supports two distinct modes: generative and discriminative. In generative mode, users can optionally provide a seed to influence the output. This tool is particularly useful for analyzing the spatial relationships and distances of objects within visual media. Hosted on Hugging Face, LOTUS Depth provides a accessible platform for researchers, developers, and enthusiasts interested in computer vision and 3D reconstruction tasks.

VLM_survey is an open-source repository offering a systematic survey of Vision-Language Models (VLMs) applied to diverse visual recognition tasks such as image classification, object detection, and semantic segmentation. It serves as a valuable resource for researchers, providing an extensive collection of academic papers, widely adopted datasets for VLM pre-training and evaluation, and categorized methodologies for VLM pre-training, transfer learning, and knowledge distillation. The repository also highlights recent advancements and future research directions in the field, making it an essential reference for anyone working with or studying VLMs.

Models Explorer is an AI tool hosted on Hugging Face Spaces, designed for discovering and exploring a wide array of AI models. It provides a platform for users to delve into model performance metrics, enabling detailed analysis and comparison of different AI models. This tool is particularly useful for individuals involved in AI research and development, offering a centralized hub to navigate the vast landscape of available models. It facilitates informed decision-making by presenting key metrics, making it easier to identify suitable models for specific applications or to benchmark existing solutions. The platform is freely accessible, promoting open exploration and collaboration within the AI community.

MoE-CAP Dashboard is an AI tool designed for analyzing and visualizing AI model performance. It allows users to generate radar plots based on various model metrics such as accuracy, cost, and throughput. The application enables users to select up to three data rows from a table to compare and visualize their performance. This tool is particularly useful for AI research and development, providing a clear and concise way to monitor and understand model behavior. It helps in identifying trends and making informed decisions regarding model optimization and selection.

MotionModel is an AI tool hosted on Hugging Face that specializes in analyzing motion within video content. It provides detailed visualizations of motion flow, neural activation, and attention, offering insights into how movement is perceived and processed. Users can upload their own videos to the platform and utilize adjustable sliders to refine their focus on particular areas of interest within the footage. This capability makes it a valuable resource for researchers and developers working with video analysis and computer vision, allowing for in-depth exploration and testing of self-attention-based motion models.

MVBench Leaderboard is a platform designed for the submission and organized display of AI model evaluation results. Users can upload JSON files containing their model's performance data, which is then integrated into a comprehensive leaderboard. This tool facilitates benchmarking by allowing researchers and developers to compare various AI models against a standardized set of metrics. It requires users to provide detailed information about their models upon submission, ensuring transparency and comparability across entries. Hosted on Hugging Face Spaces, it leverages the platform's infrastructure for accessibility and community engagement, making it a valuable resource for the AI research and development community.

My Heatmap is a data visualization tool designed to generate heatmaps using a custom model. This application provides a visual representation of data, making it easier to analyze patterns and understand complex information at a glance. It is hosted as a Hugging Face Space, indicating its accessibility and potential for community-driven development. The tool is particularly useful for anyone looking to visualize data trends and distributions through an intuitive graphical interface, aiding in data analysis and pattern recognition.

wifi-densepose, also known as RuView, is an open-source WiFi sensing platform that transforms ordinary WiFi signals into spatial intelligence. It enables real-time human pose estimation (17 COCO keypoints), vital sign monitoring (breathing and heart rate), and presence detection through walls, in the dark, and without cameras or wearables. The system leverages Channel State Information (CSI) from low-cost ESP32 sensors and can run entirely on edge hardware, integrating with Cognitum Seed for persistent memory and AI. It supports camera-free and camera-supervised training, offering high PCK@20 accuracy. The platform also generates real-time 3D point clouds by fusing camera depth, WiFi CSI, and mmWave radar, providing comprehensive environmental and human activity monitoring.

paper-rec is an AI tool designed to recommend research papers, hosted on Hugging Face. It aims to assist users in discovering relevant academic articles, making it useful for literature reviews, academic research, and staying updated on new publications. While the tool's live website currently indicates a build error, its intended purpose is to streamline the process of finding academic content. The platform is created by bluebalam and is licensed under the MIT license, suggesting an open and accessible approach to its development.

ProteinMPNN is an AI tool hosted on Hugging Face Spaces, designed for protein sequence design. It allows users to generate novel protein sequences by providing an existing protein structure, either through a PDB code or by uploading a file. The tool offers options to specify settings such as chains and sampling temperature, which influence the characteristics of the predicted sequences. This functionality is crucial for researchers and scientists in structural biology and related fields who are involved in protein engineering, drug discovery, and understanding protein function. While the current live website indicates a runtime error, the tool's core purpose is to facilitate the design of proteins with desired properties based on their structural information.

ProtGPT2_gradioFold is an AI research tool designed for protein analysis, leveraging the ProtGPT2 model to enable exploration of protein structures and folding. Hosted on Hugging Face Spaces, this tool is intended for research and educational purposes, providing a platform for scientists and students to delve into complex protein dynamics. However, at the time of review, the application is experiencing a runtime error and is currently unable to start, indicating a temporary technical issue that prevents its functionality. Once operational, it would serve as a valuable resource for academic research in bioinformatics and structural biology.

Pubky is an open-source platform offering a suite of tools and protocols designed to build a decentralized, user-controlled internet. It leverages PKARR for decentralized routing and identity management, alongside Homeservers for interoperable and censorship-resistant hosting. The platform includes Pubky App, a decentralized social media application, and Pubky Core, which provides the underlying infrastructure and SDKs for developers. Key features include identity management with Pubky Ring, a mobile key manager, and developer tools like Pubky Docker for local environment setup. Pubky also explores concepts like a Semantic Social Graph for filtering and discovery, aiming to provide a credible exit strategy for users.

Timetabled is a straightforward web application designed to help users efficiently find common free time among multiple schedules. By allowing users to input their individual timetables, the tool automatically identifies and highlights overlapping periods, making it easier to plan meetings, study sessions, or social gatherings. This eliminates the need for manual comparison and reduces the guesswork involved in coordinating schedules. The platform focuses on simplifying the process of connecting, enabling users to spend more time together rather than on logistical planning. It is ideal for academic settings, group projects, or personal use to organize events with friends and family.

French Newspapers is an AI tool hosted on Hugging Face, designed for exploring the publication dates of an extensive collection of over 3 million French newspapers and periodicals. This application provides a visual representation of publication trends, allowing users to see charts detailing how many publications were made each year. This feature is particularly useful for academic research, historical analysis, and educational purposes, offering insights into the publishing landscape of French media over time. The tool is free to use and accessible via a web interface, making it a valuable resource for anyone interested in the history and evolution of French journalism.

4d-gaussian-splatting is an open-source implementation for real-time photorealistic dynamic scene representation and rendering, based on the ICLR 2024 paper "Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting." This tool allows users to model dynamic scenes using native 4D Gaussian primitives, offering a coherent integrated approach to space and time dimensions. It builds upon the principles of 3D Gaussian Splatting and provides a dedicated rendering pipeline. The project includes resources for data preparation using datasets like DyNeRF and DNeRF, and offers scripts for training models. It's ideal for researchers and developers working on advanced 3D and animation projects.

Instruction-Tuning-Papers is a comprehensive reading list dedicated to the field of instruction tuning in language models. This resource compiles significant academic papers, tracing the evolution of this trend from foundational works like Natural-Instruction (ACL 2022), FLAN (ICLR 2022), and T0 (ICLR 2022). It serves as an invaluable reference for researchers and academics interested in how language models can be trained to better understand and execute natural language instructions, thereby enhancing their multi-task learning capabilities and generalization across unseen tasks. The repository is continuously updated with new research, offering a chronological overview of advancements in the domain.

INTIMA Companionship Benchmark Responses is a Hugging Face Space that provides a visualization tool for analyzing model responses to companionship prompts. This platform is specifically tailored for AI companionship research, allowing users to examine and compare how different AI models perform in generating companion-like interactions. While the live website currently indicates a runtime error, its intended purpose is to serve as a benchmark analysis tool for researchers and developers working on AI companionship. It aims to offer insights into the nuances of AI-generated conversational responses within this specialized domain.

Pentatonic Mode is an AI tool hosted on Hugging Face, designed to analyze short recordings (approximately 20 seconds) of Chinese music. Users can upload an audio file and select a pre-trained model. The application then processes the audio by converting it into a spectrogram, which is a visual representation of the frequencies over time. Following this, a classifier is run to identify and return the detected pentatonic modes present in the musical piece. This tool is valuable for educational purposes, musical analysis, and research into Chinese musicology, helping users understand and identify specific pentatonic scales.