Research & Education

SearchGPTool is an AI-powered search tool designed to deliver more personalized and accurate search results. By leveraging artificial intelligence, it aims to significantly improve the overall search experience for users. The tool is offered for free, making advanced search capabilities accessible to a broad audience. Its core function revolves around refining search outcomes to be more relevant to individual user needs, moving beyond traditional search engine limitations.

AI To Cards is a web-based tool designed to streamline the creation of educational flashcards. Users can input any text, and the service utilizes OpenAI's GPT-4 Turbo to automatically generate Anki-compatible flashcards. This allows for quick conversion of study materials into a format suitable for spaced repetition learning. The generated flashcards can be downloaded as a file for easy import into the Anki application, simplifying the process of creating study decks. The service offers free monthly credits, with additional credits available for purchase.

DenseFusion is an open-source code repository implementing the paper "DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion." This PyTorch-based network processes RGB-D images to predict the 6D pose of objects within a frame. It includes the full implementation of the DenseFusion model, an Iterative Refinement model, and a vanilla SegNet semantic-segmentation model. The tool is designed for tasks requiring precise object localization, such as robotic grasping experiments. It supports evaluation on both YCB_Video and LineMOD datasets and provides scripts for training and evaluation, along with pre-trained checkpoints. Users can adapt the model for their own datasets with minimal hyperparameter adjustments, provided distance metrics are in meters.

Deep3DFaceRecon_pytorch is an open-source PyTorch implementation for accurate 3D face reconstruction, building upon the original TensorFlow version. It utilizes weakly-supervised learning to reconstruct 3D faces from single images or image sets, offering improved accuracy and visual consistency. Key enhancements include a differentiable renderer using Nvdiffrast, Arcface for perceptual loss computation, and data augmentation during training. The tool achieves state-of-the-art performance on various datasets like FaceWarehouse, MICC Florence, and the NoW Challenge. It supports both inference with pre-trained models and training new models from scratch, making it suitable for researchers and developers in computer vision and 3D modeling.

DPIR (Deep Plug-and-Play Image Restoration) is an open-source project implemented in PyTorch, focusing on advanced image restoration techniques. It leverages a deep denoiser prior within a model-based framework to address various inverse problems in image processing. The tool excels in tasks such as deblurring, super-resolution, denoising, and demosaicing, offering performance that often surpasses state-of-the-art model-based methods and competes with learning-based approaches. DPIR is particularly notable for its DRUNet denoiser, which demonstrates robust performance even on extremely high, unseen noise levels, making it a powerful solution for challenging image restoration scenarios.

nnDetection is a self-configuring framework designed for 3D (volumetric) medical object detection, addressing the challenge of cumbersome method configuration in medical image analysis. Following the success of nnU-Net for image segmentation, nnDetection systematizes and automates the configuration process, allowing it to adapt to arbitrary medical detection problems without manual intervention. It achieves results comparable to or superior to state-of-the-art methods. The framework includes guides for 12 datasets used in its development and evaluation, such as ADAM and LUNA16, and supports easy integration of new datasets through a standardized input format. It is built with Python 3.8+, PyTorch, and uses Docker for easy deployment.

mvpose is an open-source project providing code for fast and robust multi-person 3D pose estimation from multiple views. Developed by zju3dv, it is based on research published in CVPR 2019 and T-PAMI 2021. The tool includes functionalities for setting up a Python environment, compiling necessary backend libraries, and preparing models and datasets for use. It supports datasets like Shelf and CampusSeq1, with detailed instructions for generating camera parameters. Users can run demos and evaluate performance on these datasets, with options to accelerate evaluation by saving predicted 2D poses and heatmaps. The project leverages components from Light head rcnn, Cascaded Pyramid Network, and CamStyle, making it a valuable resource for advanced computer vision research.

NTIRE2017 is an open-source project offering a Torch implementation of "Enhanced Deep Residual Networks for Single Image Super-Resolution." Developed by Team SNU_CVLab, it was recognized with the Best Paper Award at the CVPR 2017 Workshop (2nd NTIRE). The repository includes detailed model architectures (EDSR, MDSR), NTIRE2017 Super-resolution Challenge results, and demo and training code. Users can access trained models, information on datasets like DIV2K and Flickr2K, and super-resolution examples. The code is based on Facebook's Torch implementation of ResNet and also provides a PyTorch version for some models. It's designed for researchers and developers working on image restoration and enhancement, particularly in the field of single image super-resolution.

EagleEye is an open-source tool designed to help users find social media profiles using image recognition and reverse image search. By providing an image of a person and a clue about their name, EagleEye attempts to locate their Instagram, YouTube, Facebook, and Twitter profiles. The tool is built using Python and leverages libraries like dlib for face detection, face_recognition for dlib Python API, and Selenium for web browser automation. It requires a system with an x-server installed (Linux) and Firefox, or can be run via Docker. Users can configure the tool by placing images of the known person in a designated folder and adjusting settings in a config.json file. It's a technical tool requiring some setup for installation and usage.

drl-zh, or "Deep Reinforcement Learning: Zero to Hero!", offers a comprehensive and hands-on course designed to teach deep reinforcement learning. The curriculum is divided into two main parts: foundational concepts, where users build algorithms like DQN, SAC, and PPO from scratch, and advanced topics, which delve into areas such as curiosity-driven exploration, AlphaZero, and Reinforcement Learning with Human Feedback (RLHF). The course emphasizes learning by doing, with practical exercises ranging from playing Atari games and training robots to fine-tuning Language Models and implementing self-play with MCTS. It's structured around interactive Jupyter notebooks, providing guided TODO sections and complete solutions for reference. The entire experience is optimized for a VS Code environment, with a Dockerized setup for quick and reproducible development.

efficientdet is a PyTorch implementation of the EfficientDet object detection model, developed by Signatrix GmbH. This open-source tool provides scalable and efficient object detection capabilities, making it suitable for various computer vision tasks. It includes pre-trained weights, allowing users to get started quickly without extensive training. The repository offers scripts for training models, evaluating mean average precision (mAP) on datasets like COCO, and testing models on both datasets and video inputs. It supports Python 3.6 and PyTorch 1.2, along with other common libraries like OpenCV and TensorBoard. The implementation borrows concepts from RetinaNet, providing a robust framework for object detection research and application.

FCOS (Fully Convolutional One-Stage Object Detection) is an open-source project that provides an implementation of the FCOS algorithm for object detection. This tool is designed to completely avoid the complex computations and hyper-parameters associated with anchor boxes, offering a simpler and more efficient approach. It achieves better performance than Faster R-CNN, with significantly faster training and inference times. FCOS supports various backbones including ResNet, ResNeXt, and MobileNet, and offers models with state-of-the-art performance, reaching up to 49.0% AP on COCO test-dev. The project includes detailed instructions for installation, testing, and training, making it suitable for researchers and developers working on computer vision applications.

FAST-LIVO2 is an efficient and accurate open-source LiDAR-inertial-visual fusion localization and mapping system. It is designed for real-time 3D reconstruction and onboard robotic localization, particularly in severely degraded environments. The system integrates data from LiDAR, inertial measurement units, and visual sensors to provide robust odometry. Key features include its direct fusion approach, support for resource-constrained platforms, and an associated dataset for evaluation. The project also provides resources for building a hard-synchronized handheld device, including CAD files and source code, making it a comprehensive solution for developers working on autonomous navigation and robotics.

f2-nerf is an open-source project designed for fast neural radiance field (NeRF) training, specifically optimized for scenarios involving free camera trajectories. Built primarily on LibTorch, this tool provides a robust framework for efficient 3D scene reconstruction and novel view synthesis. Users can train F2-NeRF on custom data, including images processed with COLMAP or hloc, and generate camera poses. It also includes scripts for rendering test images and creating render paths by interpolating input camera poses. The project leverages several powerful libraries such as tiny-cuda-nn for fast MLP training, happly for PLY I/O, and eigen for linear algebra, making it a comprehensive solution for advanced NeRF applications.

FSGS, short for "Real-Time Few-Shot View Synthesis using Gaussian Splatting," is an advanced AI tool presented at ECCV 2024. It specializes in generating new views of a scene from a minimal number of input images, leveraging Gaussian Splatting technology for real-time performance. The tool provides comprehensive environmental setups, including Conda package management and CUDA 11.7 support, ensuring a robust development environment. Users can prepare data by reconstructing sparse view inputs using SfM and dense stereo matching with COLMAP, supporting datasets like LLFF and MipNeRF-360. FSGS offers clear instructions for training models with varying view counts, rendering images, and evaluating model performance, making it a valuable resource for researchers and developers in computer vision and graphics.

Genesis is a physics platform designed for general-purpose Robotics, Embodied AI, and Physical AI applications. It functions as a universal physics engine rebuilt from the ground up, capable of simulating a wide range of materials and physical phenomena. The platform is lightweight, ultra-fast, pythonic, and user-friendly, offering a powerful photo-realistic rendering system. Genesis also acts as a generative data engine, transforming natural language descriptions into various data modalities. It aims to lower the barrier to using physics simulations, unify diverse physics solvers, and automate data generation for robotics research and development.

pointnerf is an open-source implementation of Point-NeRF, a method for modeling radiance fields using neural 3D point clouds with associated neural features. This tool enables efficient rendering by aggregating neural point features near scene surfaces through a ray marching-based pipeline. A key differentiator is its ability to be initialized via direct inference of a pre-trained deep network to produce a neural point cloud, which can then be finetuned for visual quality surpassing NeRF with significantly faster training times. pointnerf also integrates with other 3D reconstruction methods and manages errors and outliers through a novel pruning and growing mechanism, making it suitable for various research applications in computer vision and graphics.

FSA-Net is an open-source research tool designed for head pose estimation from a single image, developed by Tsun-Yi Yang. Published at CVPR19, it introduces a novel approach based on regression and fine-grained feature aggregation. Unlike previous methods that often rely on landmark or depth estimation, FSA-Net aims for a more compact model by employing a soft stagewise regression scheme. A key innovation is its ability to learn fine-grained structure mapping to spatially group features before aggregation, providing part-based information and pooled values. The tool supports various face detectors like LBP, MTCNN, and SSD for robust and fast performance. It is implemented in Keras and TensorFlow, making it accessible for researchers and developers in computer vision and facial analysis.

simple-HRNet is an unofficial yet fully compatible implementation of the Deep High-Resolution Representation Learning for Human Pose Estimation paper, built with PyTorch. This tool simplifies the process of human pose estimation, offering compatibility with official pre-trained weights and delivering results consistent with the original implementation. It supports both Windows and Linux environments and includes features like multi-GPU inference, options for retrieving YOLO bounding boxes and HRNet heatmaps, and multi-person support with YOLOv3, YOLOv3-tiny, or YOLOv5. The repository also provides a live demo, scripts for training and testing on datasets like COCO, and support for TensorRT, making it a versatile solution for developers and researchers in computer vision.

instant-ngp is an open-source implementation of four neural graphics primitives: neural radiance fields (NeRF), signed distance functions (SDFs), neural images, and neural volumes. It allows users to train and render MLPs with multiresolution hash input encoding using the tiny-cuda-nn framework. The tool features an interactive GUI with comprehensive controls, including a VR mode, snapshot saving/loading, a camera path editor for videos, NeRF->Mesh and SDF->Mesh conversion, and camera pose/lens optimization. It supports various NeRF-compatible datasets and provides options for both Windows and Linux users, with Python bindings available for automated experiments.

ISCLOAM is an open-source project that implements an Intensity Scan Context based Full SLAM (Simultaneous Localization And Mapping) system, specifically designed for autonomous driving applications. This work is based on the paper "Intensity Scan Context: Coding Intensity and Geometry Relations for Loop Closure Detection" presented at ICRA 2020. The system operates at 20Hz and includes both front-end and back-end SLAM components. It provides robust localization and mapping capabilities, demonstrated through evaluations on KITTI datasets with low translation and rotation errors. The project also offers options for front-end only odometry via FLOAM and supports various Velodyne sensors.

SplaTAM is a cutting-edge system designed for Splatting, Tracking, and Mapping 3D Gaussians, enabling dense RGB-D SLAM. This tool, presented at CVPR 2024, is particularly useful for robotics and computer vision applications requiring real-time environmental understanding. Users can capture their own environments using an iPhone or LiDAR-equipped Apple device with the NeRFCapture app, and then process the data either online or offline. SplaTAM supports interactive rendering of reconstructions and allows for the export of splats to .ply files for visualization in external viewers like SuperSplat and PolyCam. It also facilitates 3D Gaussian Splatting on reconstructions and datasets with ground truth poses, making it a versatile tool for researchers and developers in the field.

Mockmate is an artificial intelligence tool designed to streamline the job interview process for both candidates and companies. For job seekers, it acts as an interview simulator, offering immediate feedback to help them practice and improve their interviewing skills. Companies can leverage Mockmate to automate initial interview stages and efficiently shortlist candidates. It utilizes natural language processing (NLP) to analyze responses, making the candidate selection process more objective and scalable.

PointRCNN is an open-source 3D object detector that directly generates accurate 3D box proposals from raw point cloud data in a bottom-up manner. It then refines these proposals using a bin-based 3D box regression loss. This tool was the first two-stage 3D object detector to use only raw point cloud as input, achieving state-of-the-art performance on the KITTI dataset at the time of its submission. PointRCNN supports features like multiple GPUs for training, GPU version rotated NMS, and faster PointNet++ inference and training. It is implemented in Python with PyTorch 1.0 and TensorboardX, making it suitable for researchers and developers in autonomous systems and computer vision.