Content & Design

RaDe-GS, or Rasterizing Depth in Gaussian Splatting, is a cutting-edge Content & Design tool developed by HKUST-SAIL. It significantly enhances the performance and accuracy of 3D scene reconstruction and rendering by incorporating advanced techniques like multi-view regularization and refined densification strategies. The project provides updated code and formulations, enabling users to achieve superior results on challenging datasets such as DTU and Tanks and Temples. It also supports novel view synthesis and geometry evaluation, making it a powerful resource for researchers and developers working with 3D Gaussian Splatting. The tool is built upon the original 3D Gaussian Splatting implementation and integrates ideas from several recent works to offer a robust and efficient solution for 3D graphics tasks.

FaceMyAI is an AI tool dedicated to generating highly realistic digital humans. These digital humans are equipped with advanced natural language processing capabilities and emotional intelligence, allowing for more natural and engaging interactions. The platform provides customizable digital assistants that can be tailored to specific needs. FaceMyAI operates on a subscription model and also offers licensing options for seamless enterprise integration. Its applications span across diverse sectors including customer service, education, healthcare, and entertainment, providing versatile solutions for businesses looking to leverage AI-powered digital human technology.

DevVerse is a technology solutions provider that previously specialized in AI-integrated web applications, offering services such as 3D modeling, machine learning, and blockchain solutions. The company aimed to empower businesses with transformative technology to fuel growth, catering to both startups and enterprises. However, the official website, devverse.org, is currently inaccessible due to an expired domain. This means that details regarding its specific features, pricing, and current offerings are unavailable. Users interested in DevVerse's services would need to wait for the domain to be renewed to access any information about its AI-powered solutions.

3dgsconverter is a Python command-line utility designed for the versatile conversion and processing of 3D Gaussian Splatting models. It offers universal N-to-N conversion capabilities, allowing users to transform models between formats such as 3DGS (.ply), KSplat, SOG, SPZ, Splat, CloudCompare, Parquet, and Compressed PLY. The tool boasts GPU acceleration via Taichi Lang, supporting NVIDIA CUDA, Apple Metal, and Vulkan/Integrated GPUs for rapid K-Means clustering and Statistical Outlier Removal (SOR). Advanced filtering options include region of interest cropping, alpha/opacity removal, density control, and SOR for cleaning up outliers. It also provides compression control to balance quality and file size, and the ability to preserve custom data elements during PLY conversions.

3DMPPE_POSENET_RELEASE is the official PyTorch implementation of the 'Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image' presented at ICCV 2019. This repository specifically focuses on the PoseNet component of the system. It offers a flexible and simple codebase compatible with various 2D and 3D, single and multi-person pose estimation datasets, including Human3.6M, MPII, MS COCO 2017, MuCo-3DHP, and MuPoTS-3D. The tool also includes visualization code for human pose estimation, making it valuable for researchers and developers working on computer vision tasks related to human understanding. Users can train and test the network, and integrate their own datasets by converting them to MS COCO format.

Compressonator is a comprehensive open-source tool suite designed for texture and 3D model compression, optimization, and analysis. It leverages the power of CPUs, GPUs, and APUs to process assets efficiently. The suite includes a graphical user interface (GUI) application for visual interaction, a command-line interface (CLI) for scripting and automation, and a software development kit (SDK) for seamless integration into existing developer toolchains. It helps artists and developers easily create compressed texture assets or optimize model meshes, while also providing tools to visualize the quality impact of various compression and rendering technologies. Compressonator supports Microsoft Windows® and Linux builds, offering block-level and high-level APIs for various compression codecs like BC1-BC7/DXTC, ETC1, ETC2, ASTC, ATC, ATI1N, and ATI2N.

Deep-high-resolution-net.pytorch is an official PyTorch implementation of the research paper "Deep High-Resolution Representation Learning for Human Pose Estimation" presented at CVPR 2019. This project focuses on maintaining high-resolution representations throughout the entire process of human pose estimation, unlike many existing methods that recover high-resolution data from low-resolution outputs. The network achieves this by starting with a high-resolution subnetwork and gradually adding parallel multi-resolution subnetworks, performing repeated multi-scale fusions. This approach leads to more accurate and spatially precise keypoint heatmaps. The repository includes code, pretrained models, and instructions for training and testing on benchmark datasets like COCO and MPII, making it a valuable resource for researchers and developers in computer vision.

Stereo-RCNN is an open-source implementation for accurate 3D object detection and estimation, primarily developed for autonomous driving applications. This tool leverages stereo images to perform simultaneous object detection and association, enhancing the precision of 3D box estimations. It also incorporates a dense alignment module for refining 3D box predictions. The project supports Pytorch 1.0.0 and Python 3.6, with a light-weight version available for scenarios with limited GPU memory. Researchers and developers can utilize Stereo-RCNN for tasks requiring robust 3D perception from image-only data, offering a valuable resource for advancing autonomous systems.

UnrealCV is an open-source project designed to bridge computer vision research with the powerful Unreal Engine (UE). It functions as a plugin for UE, extending its capabilities with a set of UnrealCV commands that enable interaction with virtual worlds. This connection facilitates communication between the Unreal Engine environment and external programs like PyTorch or TensorFlow, making it ideal for generating synthetic data for computer vision tasks. Users can either run a compiled game binary with UnrealCV embedded, requiring no prior Unreal Engine knowledge, or install the plugin directly into Unreal Engine to build new virtual worlds using the editor. It supports Unreal Engine 5.6 and offers features like optical flow image capture and calling Blueprint functions from Python.

DreamGaussian provides an official implementation for generative Gaussian splatting, a technique for efficient 3D content creation. This open-source tool allows users to generate 3D models from single images or text prompts, significantly accelerating the 3D asset pipeline. It features experimental support for advanced models like ImageDream, Stable-Zero123, and MVDream, expanding its generative capabilities. The tool includes functionalities for preprocessing images, training Gaussian splatting models, refining meshes, and visualizing the results, including 360-degree video exports. It's designed for researchers and creators looking for fast and flexible 3D model generation.

Grendel-GS is a distributed training system designed to significantly scale up 3D Gaussian Splatting training. It allows users to leverage multiple GPUs for substantially faster training times and supports a greater number of Gaussians in GPU memory, facilitating the reconstruction of larger-area, higher-resolution 3D scenes with improved PSNR. The system retains the original 3DGS algorithm, making it a direct and safe replacement for existing implementations. Grendel-GS is particularly beneficial for training large-scale 4K scenes with millions of Gaussians, offering performance improvements such as training Mip360 datasets over 3.5 times faster and completing Tanks&Temple scenes in under a minute. While it focuses on training functionality, it integrates with existing 3DGS workflows.

Deep3DFaceRecon_pytorch is an open-source PyTorch implementation for accurate 3D face reconstruction, building upon the original TensorFlow version. It utilizes weakly-supervised learning to reconstruct 3D faces from single images or image sets, offering improved accuracy and visual consistency. Key enhancements include a differentiable renderer using Nvdiffrast, Arcface for perceptual loss computation, and data augmentation during training. The tool achieves state-of-the-art performance on various datasets like FaceWarehouse, MICC Florence, and the NoW Challenge. It supports both inference with pre-trained models and training new models from scratch, making it suitable for researchers and developers in computer vision and 3D modeling.

mvpose is an open-source project providing code for fast and robust multi-person 3D pose estimation from multiple views. Developed by zju3dv, it is based on research published in CVPR 2019 and T-PAMI 2021. The tool includes functionalities for setting up a Python environment, compiling necessary backend libraries, and preparing models and datasets for use. It supports datasets like Shelf and CampusSeq1, with detailed instructions for generating camera parameters. Users can run demos and evaluate performance on these datasets, with options to accelerate evaluation by saving predicted 2D poses and heatmaps. The project leverages components from Light head rcnn, Cascaded Pyramid Network, and CamStyle, making it a valuable resource for advanced computer vision research.

nerf-pytorch is a faithful PyTorch implementation of Neural Radiance Fields (NeRF), a method renowned for achieving state-of-the-art results in synthesizing novel views of complex scenes. This open-source project successfully reproduces the original NeRF results while offering a performance improvement, running 1.3 times faster than the authors' initial TensorFlow implementation. It provides a robust framework for researchers and developers to experiment with NeRF, including tools for downloading example datasets, training models, and rendering new views. The repository also includes pre-trained models for various scenes, facilitating reproducibility and quick experimentation. It is designed for those familiar with Python and PyTorch, offering a direct path to leveraging NeRF technology.

f2-nerf is an open-source project designed for fast neural radiance field (NeRF) training, specifically optimized for scenarios involving free camera trajectories. Built primarily on LibTorch, this tool provides a robust framework for efficient 3D scene reconstruction and novel view synthesis. Users can train F2-NeRF on custom data, including images processed with COLMAP or hloc, and generate camera poses. It also includes scripts for rendering test images and creating render paths by interpolating input camera poses. The project leverages several powerful libraries such as tiny-cuda-nn for fast MLP training, happly for PLY I/O, and eigen for linear algebra, making it a comprehensive solution for advanced NeRF applications.

FSGS, short for "Real-Time Few-Shot View Synthesis using Gaussian Splatting," is an advanced AI tool presented at ECCV 2024. It specializes in generating new views of a scene from a minimal number of input images, leveraging Gaussian Splatting technology for real-time performance. The tool provides comprehensive environmental setups, including Conda package management and CUDA 11.7 support, ensuring a robust development environment. Users can prepare data by reconstructing sparse view inputs using SfM and dense stereo matching with COLMAP, supporting datasets like LLFF and MipNeRF-360. FSGS offers clear instructions for training models with varying view counts, rendering images, and evaluating model performance, making it a valuable resource for researchers and developers in computer vision and graphics.

pointnerf is an open-source implementation of Point-NeRF, a method for modeling radiance fields using neural 3D point clouds with associated neural features. This tool enables efficient rendering by aggregating neural point features near scene surfaces through a ray marching-based pipeline. A key differentiator is its ability to be initialized via direct inference of a pre-trained deep network to produce a neural point cloud, which can then be finetuned for visual quality surpassing NeRF with significantly faster training times. pointnerf also integrates with other 3D reconstruction methods and manages errors and outliers through a novel pruning and growing mechanism, making it suitable for various research applications in computer vision and graphics.

simple-HRNet is an unofficial yet fully compatible implementation of the Deep High-Resolution Representation Learning for Human Pose Estimation paper, built with PyTorch. This tool simplifies the process of human pose estimation, offering compatibility with official pre-trained weights and delivering results consistent with the original implementation. It supports both Windows and Linux environments and includes features like multi-GPU inference, options for retrieving YOLO bounding boxes and HRNet heatmaps, and multi-person support with YOLOv3, YOLOv3-tiny, or YOLOv5. The repository also provides a live demo, scripts for training and testing on datasets like COCO, and support for TensorRT, making it a versatile solution for developers and researchers in computer vision.

SplaTAM is a cutting-edge system designed for Splatting, Tracking, and Mapping 3D Gaussians, enabling dense RGB-D SLAM. This tool, presented at CVPR 2024, is particularly useful for robotics and computer vision applications requiring real-time environmental understanding. Users can capture their own environments using an iPhone or LiDAR-equipped Apple device with the NeRFCapture app, and then process the data either online or offline. SplaTAM supports interactive rendering of reconstructions and allows for the export of splats to .ply files for visualization in external viewers like SuperSplat and PolyCam. It also facilitates 3D Gaussian Splatting on reconstructions and datasets with ground truth poses, making it a versatile tool for researchers and developers in the field.

Animatable_nerf is an open-source research tool that provides the implementation for "Animatable Implicit Neural Representations for Creating Realistic Avatars from Videos," a paper accepted to TPAMI 2024 and ICCV 2021. This tool allows researchers to generate realistic avatars from video footage by leveraging animatable neural fields. It supports various configurations, including vanilla Animatable NeRF, versions with neural blend weight fields replaced by displacement fields, and versions where the canonical NeRF model is replaced with a neural surface field (SDF output). The repository includes evaluation frameworks for reconstruction quality comparison and provides access to datasets like Mobile-Stage and SyntheticHuman++ for further research and development in neural rendering and 3D human body modeling.

MVSGaussian is an open-source project designed for efficient 3D reconstruction using Gaussian Splatting from multi-view stereo (MVS) data. This tool can reconstruct unseen scenes from sparse views in a single forward pass, providing high-quality initialization for rapid training and real-time rendering. It leverages MVS to encode geometry-aware Gaussian representations and decodes them into Gaussian parameters. MVSGaussian also features a hybrid Gaussian rendering approach for novel view synthesis and a multi-view geometric consistent aggregation strategy to effectively initialize per-scene optimization. Compared to NeRF-based methods, MVSGaussian achieves superior view synthesis quality with reduced training computational costs and real-time rendering speeds, making it valuable for computer vision research and 3D modeling applications.

Instruct-NeRF2NeRF is an open-source tool designed for editing 3D scenes with natural language instructions, building on the Nerfstudio framework. It enables users to modify Neural Radiance Fields (NeRF) scenes by providing textual prompts, offering a powerful way to interact with and transform 3D environments. The tool requires users to first train a regular nerfacto scene with their data, then apply Instruct-NeRF2NeRF for editing. It supports various configurations to balance memory usage and quality, including full, small, and tiny models. Users can specify prompts and guidance scales for the editing process. The project also provides an extension for Gaussian Splatting called Instruct-GS2GS, demonstrating its extensibility.

Neurecon is an open-source project offering unofficial PyTorch implementations of advanced neural rendering techniques for multi-view 3D reconstruction. It focuses on unifying neural implicit surfaces and radiance fields, as seen in papers like UNISURF, NeuS, and VolSDF. The tool allows users to reconstruct 3D surfaces and appearance from pure posed RGB images, without requiring masks, depths, or ground truth meshes. It leverages volume rendering to efficiently learn rough shapes early in training and then refines fine details, bridging the gap between implicit 3D surfaces and volume rendering. Neurecon is a valuable resource for researchers and developers exploring the cutting edge of 3D reconstruction.

stack-chan is an open-source project featuring a JavaScript-driven robot embedded in M5Stack. This super-kawaii robot can display a range of cute faces and expressions, including happy, angry, and sad. Users have the flexibility to customize the robot's face and expressions, as well as add various M5Units for enhanced functionality. The project provides all necessary components, including firmware source codes, stereolithography (STL) files for the case, and schematics with board layout data. It supports driving serial (TTL) and PWM servos and encourages users to develop their own applications. The project is distributed under the Apache version 2.0 license, making it accessible for developers and hobbyists.