Research & Education

Prithvi 100M Burn Scars Demo is a specialized AI application designed for the detection of burn scars using HLS geotiff images. Developed by ibm-nasa-geospatial, this tool enables users to upload their own images, provided they contain specific channels in reflectance units. The application then processes these images to identify and highlight burn scars, outputting a color composite image as a result. This demonstration tool is part of the IBM-NASA Prithvi Models Family, showcasing capabilities in geospatial data analysis and AI model application for environmental monitoring.

imbalanced-semi-self is an open-source GitHub repository offering implementation code for the paper "Rethinking the Value of Labels for Improving Class-Imbalanced Learning" presented at NeurIPS 2020. This tool focuses on enhancing performance on imbalanced (long-tailed) datasets by utilizing both semi-supervised learning (with unlabeled data) and self-supervised pre-training. It demonstrates how these techniques can improve class separation and mitigate tail class leakage, even with varying imbalanceness in labeled and unlabeled data. The repository includes code for training models with extra unlabeled data, self-supervised pre-training using Rotation prediction or MoCo, and network training with SSP models, supporting datasets like CIFAR, SVHN, ImageNet-LT, and iNaturalist 2018. It provides detailed instructions for installation, data preparation, pseudo-label generation, and testing pre-trained checkpoints.

Science Leaderboard is a platform designed to evaluate and compare the science reasoning capabilities of various AI models. It presents and refreshes leaderboard data in a table format, offering a clear overview of model performance. Users can access detailed information about the models and contribute new results by submitting JSON files. This tool is particularly useful for researchers and developers in the AI community who need to benchmark their models against others in the field, identify top-performing AI systems, and track advancements in science-related AI applications.

TCD serves as the official demonstration space for Trajectory Consistency Distillation (TCD), a cutting-edge technique in AI research. Hosted on Hugging Face Spaces, this tool is designed for researchers and academics to interact with and understand the principles behind TCD. While the current live demo encountered a runtime error related to a missing PEFT backend, the underlying purpose is to showcase the application and potential of trajectory consistency distillation. This platform is intended to facilitate exploration and learning for those interested in advanced AI model optimization and distillation methods.

MuJoCo Playground is an open-source library developed by Google DeepMind, offering a comprehensive suite of GPU-accelerated environments for advanced robot learning research and sim-to-real transfer. Built with MuJoCo MJX, it includes classic control environments from dm_control, quadruped and bipedal locomotion environments, and non-prehensile and dexterous manipulation environments. The library also features vision-based support via the MJWarp Batch Renderer. It supports training with both the MuJoCo MJX JAX implementation and the MuJoCo Warp implementation, making it a versatile tool for developers and researchers in robotics.

VLM R1 Referral Expression is an AI tool designed for referral expression tasks, allowing users to upload an image and provide a descriptive text. The application then identifies and highlights the specific region within the image that corresponds to the provided description. A key feature of this tool is its ability to display the reasoning process behind its selections, offering transparency into how the AI interprets the input and makes its visual correlations. This functionality makes it particularly useful for understanding AI model behavior in computer vision tasks. While the tool's live website currently shows a runtime error related to NVIDIA driver issues, its intended purpose is to provide visual explanations for descriptive queries.

WebGPU Depth Anything V2 is an advanced AI tool designed for estimating depth in images. Users can upload an image to generate a detailed depth map, which visually represents the distance of objects within the scene. This tool leverages WebGPU technology, suggesting potential for efficient processing directly within a web browser. It serves as an updated iteration of the original Depth Anything model, likely incorporating improvements in accuracy, performance, or features. This capability is particularly valuable for researchers and developers in computer vision, enabling applications that require precise depth information for tasks such as 3D reconstruction, scene understanding, or robotics.

WebGPU Real-time Depth Estimation is an AI tool designed for real-time depth estimation from webcam video, leveraging WebGPU technology. This application provides a dynamic 3D-like view of your surroundings, making it suitable for interactive applications and research in computer vision. Users can adjust parameters such as stream scale and image size to optimize the balance between processing speed and visual detail. This capability is particularly useful for developers and researchers who require rapid depth map generation for their projects, enabling them to explore and implement real-time computer vision solutions efficiently. The tool's focus on real-time performance and adjustable settings makes it a valuable asset for experimental and practical applications in depth sensing.

PlayCanvas Simulation Vehicle Physics offers a live HTML5-based vehicle physics simulation, allowing users to experience realistic vehicle dynamics directly in their browser. This tool, hosted on Hugging Face Spaces, enables interactive driving with WASD controls. It's designed for those interested in exploring or demonstrating physics simulations without the need for complex software installations. The simulation provides a smooth and responsive experience, making it suitable for educational purposes, rapid prototyping, or simply for engaging with interactive physics models. Its accessibility through a web browser makes it a convenient platform for quick experimentation and visualization of vehicle mechanics.

Latex Ocr is a specialized tool engineered to transform images containing mathematical formulas and equations directly into Latex code. This functionality is particularly beneficial for users who frequently work with academic or scientific documents. By enabling the extraction and digitization of complex mathematical expressions from visual sources, Latex Ocr streamlines the process of incorporating these elements into Latex-based projects. It serves as a valuable resource for individuals in educational and research fields.

Complex-YOLOv4-Pytorch offers a robust PyTorch implementation of the Complex-YOLOv4 paper, focusing on real-time 3D object detection using point clouds. This tool is designed for researchers and developers working with LiDAR data, providing features like distributed data parallel training for efficiency and Tensorboard integration for monitoring training progress. It incorporates advanced augmentation techniques such as Mosaic/Cutout for training and utilizes GIoU loss for optimizing rotated bounding boxes, enhancing detection accuracy. The project also highlights an anchor-free approach, faster training and inference, and eliminates the need for Non-Max-Suppression, making it a powerful solution for 3D object detection tasks.

DAMO-YOLO is a fast and accurate open-source object detection method developed by the TinyML Team from Alibaba DAMO Data Analytics and Intelligence Lab. It extends the YOLO series with new technologies including Neural Architecture Search (NAS) backbones, efficient Reparameterized Generalized-FPN (RepGFPN), a lightweight head with AlignedOTA label assignment, and distillation enhancement. The tool achieves higher performance than state-of-the-art YOLO series and provides not only powerful models but also highly efficient training strategies and complete tools from training to deployment. It supports various models, including general, light, and 701-category models, and offers tutorials for custom dataset finetuning and TensorRT Int8 Quantization.

img2pose is an open-source PyTorch implementation for real-time, six degrees of freedom (6DoF), 3D face pose estimation. This tool uniquely performs face alignment and detection without requiring preliminary face detection or facial landmark localization, simplifying the process. It leverages a Faster R-CNN-based model to regress 6DoF pose for all faces in a photo, even tiny ones. The system allows for visualization of detections, customization of projected bounding boxes, and cropping/aligning faces for further processing. Accepted at CVPR 2021, img2pose outperforms state-of-the-art face pose estimators and even surpasses comparable models on the WIDER FACE detection benchmark, despite not being optimized for bounding box labels.

RaDe-GS, or Rasterizing Depth in Gaussian Splatting, is a cutting-edge Content & Design tool developed by HKUST-SAIL. It significantly enhances the performance and accuracy of 3D scene reconstruction and rendering by incorporating advanced techniques like multi-view regularization and refined densification strategies. The project provides updated code and formulations, enabling users to achieve superior results on challenging datasets such as DTU and Tanks and Temples. It also supports novel view synthesis and geometry evaluation, making it a powerful resource for researchers and developers working with 3D Gaussian Splatting. The tool is built upon the original 3D Gaussian Splatting implementation and integrates ideas from several recent works to offer a robust and efficient solution for 3D graphics tasks.

TotalSegmentator is a powerful tool designed for robust segmentation of over 100 important anatomical structures within both CT and MR images. It has been extensively trained on a diverse dataset, encompassing various scanners, institutions, and protocols, ensuring its effectiveness across a broad spectrum of medical imaging data. The tool supports a wide array of subtasks, including detailed segmentation of lung vessels, body parts, vertebrae, cerebral bleeds, hip implants, and various head and neck structures. It is available for use on Ubuntu, Mac, and Windows, supporting both CPU and GPU operations. While not intended for clinical usage as a standalone medical device, it is certified as a component within several FDA-approved products.

The Temporal Shift Module (TSM) is an open-source PyTorch implementation designed for efficient video understanding. It allows for temporal modeling in video analysis tasks, such as action recognition, by shifting part of the channels along the temporal dimension. TSM is a plug-and-play module that adds zero parameters and zero FLOPs, making it highly efficient. The project provides pre-trained models on datasets like Kinetics-400 and Something-Something, along with code for data preparation, testing, and training. It also features a live demo for online hand gesture recognition on NVIDIA Jetson Nano, showcasing its real-time capabilities.

3DMPPE_POSENET_RELEASE is the official PyTorch implementation of the 'Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image' presented at ICCV 2019. This repository specifically focuses on the PoseNet component of the system. It offers a flexible and simple codebase compatible with various 2D and 3D, single and multi-person pose estimation datasets, including Human3.6M, MPII, MS COCO 2017, MuCo-3DHP, and MuPoTS-3D. The tool also includes visualization code for human pose estimation, making it valuable for researchers and developers working on computer vision tasks related to human understanding. Users can train and test the network, and integrate their own datasets by converting them to MS COCO format.

LuaRadio is a lightweight and embeddable flow graph signal processing framework specifically designed for software-defined radio (SDR). Built on LuaJIT, it offers a small binary footprint and no external hard dependencies, making it highly portable. The framework provides a comprehensive suite of source, sink, and processing blocks, along with a simple API for defining and running flow graphs, creating custom blocks, and managing data types. It's ideal for rapidly prototyping software radios, developing modulation/demodulation utilities, and conducting signal processing experiments. LuaRadio can also be embedded into existing radio applications, serving as a user-scriptable engine for advanced signal processing tasks. It supports computational acceleration through LuaJIT's FFI to wrap external libraries like VOLK, liquid-dsp, and others, ensuring efficient performance.

Rofunc is an open-source Python package designed for robot learning from demonstration and robot manipulation. It provides a comprehensive framework for developing and deploying advanced robot learning algorithms. The tool is hosted on GitHub, making it accessible for researchers and developers in the robotics field. Rofunc facilitates the entire workflow, from initial algorithm development to practical deployment, supporting various aspects of robot control and interaction. Its open-source nature encourages community contributions and collaborative development, making it a valuable resource for advancing robotics research and applications.

Stereo-RCNN is an open-source implementation for accurate 3D object detection and estimation, primarily developed for autonomous driving applications. This tool leverages stereo images to perform simultaneous object detection and association, enhancing the precision of 3D box estimations. It also incorporates a dense alignment module for refining 3D box predictions. The project supports Pytorch 1.0.0 and Python 3.6, with a light-weight version available for scenarios with limited GPU memory. Researchers and developers can utilize Stereo-RCNN for tasks requiring robust 3D perception from image-only data, offering a valuable resource for advancing autonomous systems.

vedadet is a single-stage object detection toolbox built on PyTorch, offering a modular design that re-engineers MMDetection for enhanced flexibility and deployment. It decomposes the detector into four key parts: data pipeline, model, postprocessing, and criterion, making it straightforward to convert PyTorch models into TensorRT engines. This design facilitates efficient deployment on NVIDIA devices such as Tesla V100, Jetson Nano, and Jetson AGX Xavier. The toolbox supports several popular single-stage detectors, including RetinaNet and FCOS, right out of the box. Its friendly integration with TensorRT allows for easy model conversion and deployment through both Python and C++ front-ends, making it a powerful tool for developers working on object detection tasks.

TempestV0.1 GPU Demo is a demonstration of AI capabilities, specifically designed to showcase the TempestV0.1 model. Hosted on Hugging Face Spaces, this tool leverages GPU processing to provide a platform for users to explore and test the model's functionalities. While currently paused, it aims to offer insights into advanced AI applications. Users interested in utilizing this Space are encouraged to contact the author through the community tab to request its restart, indicating its potential for academic research and educational purposes.

DenseFusion is an open-source code repository implementing the paper "DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion." This PyTorch-based network processes RGB-D images to predict the 6D pose of objects within a frame. It includes the full implementation of the DenseFusion model, an Iterative Refinement model, and a vanilla SegNet semantic-segmentation model. The tool is designed for tasks requiring precise object localization, such as robotic grasping experiments. It supports evaluation on both YCB_Video and LineMOD datasets and provides scripts for training and evaluation, along with pre-trained checkpoints. Users can adapt the model for their own datasets with minimal hyperparameter adjustments, provided distance metrics are in meters.

nnDetection is a self-configuring framework designed for 3D (volumetric) medical object detection, addressing the challenge of cumbersome method configuration in medical image analysis. Following the success of nnU-Net for image segmentation, nnDetection systematizes and automates the configuration process, allowing it to adapt to arbitrary medical detection problems without manual intervention. It achieves results comparable to or superior to state-of-the-art methods. The framework includes guides for 12 datasets used in its development and evaluation, such as ADAM and LUNA16, and supports easy integration of new datasets through a standardized input format. It is built with Python 3.8+, PyTorch, and uses Docker for easy deployment.