Research & Education

efficientdet is a PyTorch implementation of the EfficientDet object detection model, developed by Signatrix GmbH. This open-source tool provides scalable and efficient object detection capabilities, making it suitable for various computer vision tasks. It includes pre-trained weights, allowing users to get started quickly without extensive training. The repository offers scripts for training models, evaluating mean average precision (mAP) on datasets like COCO, and testing models on both datasets and video inputs. It supports Python 3.6 and PyTorch 1.2, along with other common libraries like OpenCV and TensorBoard. The implementation borrows concepts from RetinaNet, providing a robust framework for object detection research and application.

FCOS (Fully Convolutional One-Stage Object Detection) is an open-source project that provides an implementation of the FCOS algorithm for object detection. This tool is designed to completely avoid the complex computations and hyper-parameters associated with anchor boxes, offering a simpler and more efficient approach. It achieves better performance than Faster R-CNN, with significantly faster training and inference times. FCOS supports various backbones including ResNet, ResNeXt, and MobileNet, and offers models with state-of-the-art performance, reaching up to 49.0% AP on COCO test-dev. The project includes detailed instructions for installation, testing, and training, making it suitable for researchers and developers working on computer vision applications.

FAST-LIVO2 is an efficient and accurate open-source LiDAR-inertial-visual fusion localization and mapping system. It is designed for real-time 3D reconstruction and onboard robotic localization, particularly in severely degraded environments. The system integrates data from LiDAR, inertial measurement units, and visual sensors to provide robust odometry. Key features include its direct fusion approach, support for resource-constrained platforms, and an associated dataset for evaluation. The project also provides resources for building a hard-synchronized handheld device, including CAD files and source code, making it a comprehensive solution for developers working on autonomous navigation and robotics.

Genesis is a physics platform designed for general-purpose Robotics, Embodied AI, and Physical AI applications. It functions as a universal physics engine rebuilt from the ground up, capable of simulating a wide range of materials and physical phenomena. The platform is lightweight, ultra-fast, pythonic, and user-friendly, offering a powerful photo-realistic rendering system. Genesis also acts as a generative data engine, transforming natural language descriptions into various data modalities. It aims to lower the barrier to using physics simulations, unify diverse physics solvers, and automate data generation for robotics research and development.

FSA-Net is an open-source research tool designed for head pose estimation from a single image, developed by Tsun-Yi Yang. Published at CVPR19, it introduces a novel approach based on regression and fine-grained feature aggregation. Unlike previous methods that often rely on landmark or depth estimation, FSA-Net aims for a more compact model by employing a soft stagewise regression scheme. A key innovation is its ability to learn fine-grained structure mapping to spatially group features before aggregation, providing part-based information and pooled values. The tool supports various face detectors like LBP, MTCNN, and SSD for robust and fast performance. It is implemented in Keras and TensorFlow, making it accessible for researchers and developers in computer vision and facial analysis.

ISCLOAM is an open-source project that implements an Intensity Scan Context based Full SLAM (Simultaneous Localization And Mapping) system, specifically designed for autonomous driving applications. This work is based on the paper "Intensity Scan Context: Coding Intensity and Geometry Relations for Loop Closure Detection" presented at ICRA 2020. The system operates at 20Hz and includes both front-end and back-end SLAM components. It provides robust localization and mapping capabilities, demonstrated through evaluations on KITTI datasets with low translation and rotation errors. The project also offers options for front-end only odometry via FLOAM and supports various Velodyne sensors.

SplaTAM is a cutting-edge system designed for Splatting, Tracking, and Mapping 3D Gaussians, enabling dense RGB-D SLAM. This tool, presented at CVPR 2024, is particularly useful for robotics and computer vision applications requiring real-time environmental understanding. Users can capture their own environments using an iPhone or LiDAR-equipped Apple device with the NeRFCapture app, and then process the data either online or offline. SplaTAM supports interactive rendering of reconstructions and allows for the export of splats to .ply files for visualization in external viewers like SuperSplat and PolyCam. It also facilitates 3D Gaussian Splatting on reconstructions and datasets with ground truth poses, making it a versatile tool for researchers and developers in the field.

MVSGaussian is an open-source project designed for efficient 3D reconstruction using Gaussian Splatting from multi-view stereo (MVS) data. This tool can reconstruct unseen scenes from sparse views in a single forward pass, providing high-quality initialization for rapid training and real-time rendering. It leverages MVS to encode geometry-aware Gaussian representations and decodes them into Gaussian parameters. MVSGaussian also features a hybrid Gaussian rendering approach for novel view synthesis and a multi-view geometric consistent aggregation strategy to effectively initialize per-scene optimization. Compared to NeRF-based methods, MVSGaussian achieves superior view synthesis quality with reduced training computational costs and real-time rendering speeds, making it valuable for computer vision research and 3D modeling applications.

motpy is a Python library designed for multi-object tracking using the tracking-by-detection paradigm. It offers a straightforward yet robust baseline for developers to implement object tracking without needing to build the entire algorithmic stack from scratch. Key features include IOU and optional feature similarity matching, Kalman filters for modeling object trackers, and configurable system orders for object position and size. The library is optimized for performance, achieving real-time tracking even on resource-constrained devices like the Raspberry Pi. It supports various use cases, from synthetic 2D tracking to detecting and tracking objects in videos and webcam face tracking, making it a versatile tool for computer vision applications.

safe-control-gym offers physics-based CartPole and Quadrotor Gym environments built using PyBullet, featuring symbolic a priori dynamics powered by CasADi. This framework is designed for learning-based control, as well as model-free and model-based reinforcement learning (RL). It includes symbolic safety constraints and implements input, parameter, and dynamics disturbances to rigorously test the robustness and generalizability of various control approaches. The tool provides a unified benchmark suite for safe learning-based control and RL in robotics, supporting a range of implemented controllers like PID, LQR, iLQR, MPC, SAC, and PPO, alongside safety filters such as MPSC and CBF. It also offers performance comparisons against other popular Gym environments.

VideoSuperResolution is an open-source project offering a comprehensive collection of state-of-the-art video and single-image super-resolution architectures. These models are reimplemented in TensorFlow, with several referenced PyTorch implementations also included. The project provides a simple, easy-to-use framework for training and data processing based on TensorFlow, capable of handling raw NV12/YUV as well as sequences of images as inputs. Users can install the package via PyPI and download pre-trained weights for various models like SRCNN, VESPCN, and ESRGAN. It supports a wide range of datasets for training and testing, making it a valuable resource for researchers and developers working on image and video enhancement.

voc-dpm is an open-source object detection system, specifically voc-release5, developed by Ross Girshick. It implements object detection based on mixtures of deformable part models (DPMs) and supports both binary latent SVM and weak-label structural SVM (WL-SSVM) for learning. The system includes pretrained models for PASCAL and INRIA Person datasets, along with features like context rescoring and the star-cascade detection algorithm. Implemented primarily in MATLAB with MEX C++ helper functions for efficiency, it requires MATLAB, GCC, and at least 4GB of memory. The GitHub repository serves as a code release, with the author recommending checking their website for the latest, more thoroughly tested tarball.

Yolov7-tracker is a comprehensive toolbox designed for multi-object tracking, implementing the tracking-by-detection paradigm. It supports a wide range of Yolo detectors, from YOLOX to YOLO v12 by ultralytics, and integrates numerous advanced trackers including SORT, DeepSORT, ByteTrack, BoT-SORT, OCSORT, Strong SORT, and more. The tool is built with a unified code style and modular design, decoupling the detector, tracker, ReID model, and Kalman filter, which simplifies experimentation and integration into custom projects. It also supports TensorRT for optimized inference and offers ReID models for both pedestrian and vehicle re-identification. The toolbox is compatible with datasets like MOT17 and VisDrone2019, providing detailed instructions for data preparation and training.

Psmoveapi is a versatile, cross-platform library designed for 6DoF (six degrees of freedom) tracking of the PlayStation Move Motion Controller. It integrates advanced sensor fusion and computer vision techniques to provide precise positional and rotational tracking. The library also extends its functionality to include ambient display control through the PS Move's LED orb, enhancing user feedback and immersion. Developers can utilize psmoveapi to gain direct PC access to the PS Move controller, facilitating communication via both Bluetooth and USB connections. This makes it an ideal tool for creating custom applications, games, or research projects that leverage the unique input capabilities of the PS Move controller on various computing platforms.

R-FCN (Region-based Fully Convolutional Networks) is an open-source object detection framework designed for computer vision research and applications. It utilizes deep fully-convolutional networks to achieve accurate and efficient object detection. Unlike previous region-based detectors that apply costly per-region sub-networks, R-FCN shares almost all computation on the entire image, making it highly efficient. The framework can integrate powerful fully convolutional image classifier backbones, such as ResNets, for enhanced performance. It supports end-to-end training and inference for object detection and has been tested on Windows and Ubuntu platforms, requiring MATLAB and a Caffe build.

sdfstudio is a unified, open-source framework designed for neural implicit surface reconstruction, leveraging the foundation of the Nerfstudio project. It provides a modular architecture that allows for the implementation and exploration of different surface reconstruction methods, such as UniSurf, VolSDF, and NeuS. The framework supports various scene representations and datasets, making it a versatile tool for advanced 3D modeling, research, and development in the field of neural implicit surfaces. Its open-source nature encourages community contributions and provides a flexible platform for experimenting with cutting-edge 3D reconstruction techniques.

ssm is a powerful tool designed for Bayesian learning and inference within state space models. It offers comprehensive functionalities for simulating, learning, and performing inference across a variety of state space models. The project is currently undergoing a JAX refactor, which aims to leverage JIT compilation and provide enhanced support for GPU and TPU hardware, significantly boosting performance and computational efficiency for complex scientific computing tasks. This makes ssm particularly valuable for researchers and data scientists working with dynamic systems and requiring robust statistical modeling capabilities.

taichi_3d_gaussian_splatting is an unofficial, open-source implementation of 3D Gaussian Splatting, designed for real-time radiance field rendering. This tool utilizes the Taichi programming language, known for its high-performance computing capabilities, to process and render complex 3D scenes efficiently. It takes multiple-view images, a sparse point cloud, and camera pose as input to train and optimize the point cloud representation. This allows for the creation of highly detailed and realistic 3D environments that can be rendered in real-time, making it suitable for applications requiring interactive 3D visualization or rapid scene generation.

AlphaPose is a robust, open-source system designed for real-time and accurate full-body multi-person pose estimation and tracking. It stands out as one of the first open-source systems to achieve high mAP scores on COCO and MPII datasets. The tool also incorporates an efficient online pose tracker called Pose Flow, which excels in matching poses across frames. Key features include support for COCO 17 keypoints, Halpe 26 and 136 keypoints with tracking, and SMPL integration for 3D pose and shape estimation. AlphaPose is compatible with both Linux and Windows, and a Jittor version is available, offering significant speed improvements during the training stage. It is ideal for researchers and developers working on computer vision projects requiring precise human pose analysis.

Aircrack-ng is a comprehensive, open-source suite of command-line tools designed for assessing and auditing WiFi network security. It focuses on various aspects of WiFi security, including monitoring for packet capture and data export, attacking through replay attacks and deauthentication, testing WiFi card capabilities, and cracking WEP and WPA PSK (WPA 1 and 2) passwords. Its command-line interface allows for extensive scripting, making it a favorite among security professionals and developers. While primarily developed for Linux, it also supports Windows, macOS, FreeBSD, OpenBSD, NetBSD, Solaris, and eComStation 2, offering broad compatibility for security assessments.

torch-ngp offers a PyTorch CUDA extension implementation of instant-ngp, supporting both Signed Distance Functions (SDF) and Neural Radiance Fields (NeRF). It includes a graphical user interface (GUI) for training and visualization. The repository also features PyTorch implementations of TensoRF, adapted to instant-ngp's NeRF framework, and CCNeRF for compressible-composable NeRF via rank-residual decomposition. Additionally, it provides an implementation of D-NeRF for dynamic scenes and experimental features like text-guided NeRF editing. The tool supports various datasets and offers options for different backbones and optimization techniques, making it a versatile platform for neural graphics research.

HDM is an AI tool hosted on Hugging Face that specializes in the template-free reconstruction of human-object interaction. It leverages procedural interaction generation to achieve its results, making it a valuable resource for researchers and developers in the field of computer vision and human-computer interaction. The tool is designed to facilitate advanced studies and applications related to how humans interact with objects, offering a flexible and accessible platform for experimentation and development. Its availability as a free template on Hugging Face further enhances its utility for academic and research purposes.

UAV-path-planning is an open-source project hosted on GitHub that focuses on multi and single unmanned aerial vehicle (UAV) path planning using deep reinforcement learning. The repository offers code implementations for different scenarios, including multi-UAV collaborative path planning based on the MASAC reinforcement learning algorithm and single UAV path planning using maximum entropy safe reinforcement learning. It is built on the PyTorch framework and provides detailed instructions on how to use the code, along with video tutorials in Chinese for both multi-UAV and single UAV path planning. The project acknowledges contributions from various open-source communities and individuals, making it a valuable resource for researchers and developers in the field of drone autonomy.

Monocular depth estimation is a specialized tool designed for computer vision tasks, specifically focusing on inferring depth information from a single 2D image. This capability is crucial for various applications in computer vision, including 3D scene understanding, object recognition, and autonomous navigation. By analyzing visual cues within a single image, the tool aims to reconstruct the spatial relationships and distances of objects in the scene. While the current live website indicates a runtime error, the underlying purpose of such a tool is to provide researchers and developers with a method to extract valuable 3D data from readily available 2D imagery, facilitating advancements in areas requiring spatial awareness.