Research & Education

Spleen 3D Segmentation With MONAI is an AI-powered application hosted on Hugging Face Spaces, designed for medical image analysis. This tool allows users to upload a 3D medical image containing a spleen, and it will process the image to generate a segmented output. The segmentation highlights the spleen, making it easier for medical professionals to analyze its structure and identify potential issues. Built with MONAI, a PyTorch-based framework for deep learning in healthcare imaging, this tool demonstrates the application of AI in assisting diagnostics and research within the medical domain. While the current live website indicates a runtime error, the intended functionality is to provide a clear, segmented view of the spleen from complex 3D medical scans.

segmentation_models.pytorch is an Open Source Python library designed for semantic image segmentation using PyTorch. It provides a high-level API that allows users to create neural networks with minimal code, supporting 12 encoder-decoder model architectures such as Unet, Unet++, Segformer, and DPT. The library boasts an extensive collection of over 800 pretrained convolutional and transformer-based encoders, including timm support, which helps achieve faster and more stable convergence during training. It also includes popular metrics and losses for training routines, such as Dice and Jaccard, and is compatible with ONNX export and torch script/trace/compile. This makes it a versatile tool for researchers and practitioners in computer vision.

Focoos AI reshapes computer vision by offering ultra-efficient models designed to reduce costs, automate hardware integration, and ensure peak performance across various devices. The platform allows ML Engineers to train, deploy, and iterate models faster than ever, supporting both cloud and edge environments. Its models are engineered for speed, delivering up to 10x faster inference and being 4x lighter in compute and memory compared to mainstream alternatives. Focoos AI provides pre-trained, production-ready models that can be instantly deployed and easily fine-tuned. It features an all-in-one platform for managing, comparing, monitoring, and deploying models, alongside an open-source library for community collaboration and local use. The tool emphasizes security, control, and sustainability, making it suitable for applications in manufacturing, smart cities, and autonomous systems.

TensorLayer is a powerful, open-source deep learning and reinforcement learning library built for scientists and engineers. It offers an extensive collection of customizable neural layers, enabling rapid development of advanced AI models. Inspired by PyTorch, TensorLayer provides transparent and flexible APIs, making it easier to build and train complex AI models compared to other TensorFlow wrappers. It supports multiple backends including TensorFlow, PyTorch, MindSpore, PaddlePaddle, OneFlow, and Jittor, allowing deployment on various hardware like Nvidia-GPU and Huawei-Ascend. The library is recognized for its simplicity, flexibility, and high performance, with comprehensive documentation and a large community.

tiefvision is an integrated end-to-end image-based search engine powered by deep learning. It offers comprehensive functionalities including image classification, image location (based on OverFeat), and image similarity (based on Deep Ranking). The system is built using Torch for its deep learning modules and the Play Framework (Scala version) for its tooling modules. It currently supports Linux operating systems with CUDA-enabled GPUs, indicating a focus on performance-intensive image processing tasks. Beyond its core deep learning capabilities, tiefvision also provides a suite of web tools designed to streamline dataset generation and enhance productivity, such as visual database editors and automated dataset generation for training and testing.

Recursion is a clinical-stage TechBio company dedicated to decoding biology through AI to radically improve lives. Founded over a decade ago, Recursion utilizes its proprietary Recursion OS, an AI-native, end-to-end drug discovery and development platform. This platform integrates biology, chemistry, and clinical development into a unified intelligence system, powered by multimodal data, purpose-built AI models, and bilingual teams. Recursion aims to reduce the massive 90% failure rate of traditional drug discovery by using AI to understand cellular disruptions driving disease. The company has yielded an advanced pipeline of potential first-in-class and best-in-class treatments for conditions with high unmet need, including aggressive cancers and rare diseases, demonstrating significant improvements in speed, efficiency, and reduced costs from hit identification to IND-enabling studies.

MeshCNN is a general-purpose deep neural network specifically designed for 3D triangular meshes, implemented using PyTorch. This framework enables advanced tasks such as 3D shape classification and segmentation by applying convolutional, pooling, and unpooling layers directly on the mesh edges. It offers a robust solution for researchers and developers working with 3D data, providing a novel approach to process geometric information. The repository includes scripts for installation, training, and testing on datasets like SHREC and Humans, making it accessible for practical application and further development in the field of geometric deep learning.

Mocha.jl is a deep learning framework for the Julia programming language, drawing inspiration from the C++ framework Caffe. Although now deprecated, it was designed for efficient training of deep and shallow convolutional neural networks, supporting optional unsupervised pre-training via stacked auto-encoders. The framework boasts a modular architecture with isolated components for layers, activation functions, solvers, and more, allowing for easy extension. Written in Julia, it offers a high-level interface for intuitive deep neural network experimentation. Mocha.jl provides multiple backends, including a portable pure Julia backend, a faster native extension backend, and a highly efficient GPU backend utilizing NVidia® cuDNN and CUDA kernels. It also supports HDF5 for data and model storage, ensuring compatibility with other computational tools, and can import Caffe model snapshots.

rnn is a specialized library designed for building Recurrent Neural Networks within the Torch7's nn framework. It offers functionalities to construct different types of RNN architectures, including LSTMs (Long Short-Term Memory), GRUs (Gated Recurrent Units), and BRNNs (Bidirectional Recurrent Neural Networks). This tool is particularly useful for developers and researchers working on deep learning projects that require sequential data processing and advanced neural network models. While the original repository is deprecated, its principles and functionalities laid a foundation for subsequent RNN implementations in Torch.

SuperGluePretrainedNetwork is a research project from Magic Leap, presented at CVPR 2020, focusing on learning feature matching using Graph Neural Networks. The core of the project is the SuperGlue network, which integrates a Graph Neural Network with an Optimal Matching layer. This architecture is specifically designed to perform matching tasks on two distinct sets of sparse image features. The repository offers both the PyTorch code implementation and pretrained weights, making it accessible for researchers and developers interested in computer vision and feature matching applications. It serves as a valuable resource for those looking to implement or build upon advanced feature matching techniques.

StellarGraph is a comprehensive Python library designed for machine learning on various types of graphs and networks. It provides a rich collection of state-of-the-art algorithms, including GraphSAGE, GCN, GAT, Node2Vec, and Metapath2Vec, enabling users to perform tasks such as representation learning for nodes and edges, classification of nodes or entire graphs, and link prediction. The library supports diverse graph structures, from homogeneous to heterogeneous and knowledge graphs, and integrates seamlessly with TensorFlow 2, Keras, Pandas, and NumPy. This makes it user-friendly, modular, and extensible, allowing for smooth interoperability with existing machine learning workflows and easy augmentation of its core algorithms.

sumo-rl is an open-source tool designed to simplify the creation and management of Reinforcement Learning (RL) environments for Traffic Signal Control using SUMO. It offers a straightforward interface, ensuring compatibility with widely used RL libraries and frameworks such as Gymnasium, PettingZoo, stable-baselines3, and RLlib. The tool supports both single-agent and multi-agent RL scenarios, allowing for flexible experimentation. Users can easily customize observation spaces and reward functions to suit their specific research or application needs. sumo-rl is particularly useful for developers and researchers focused on advancing AI agents for traffic management and optimization, providing a robust platform for simulating and evaluating different control strategies.

torchscale is a PyTorch library specifically engineered to facilitate the scaling of Transformer models, which are fundamental to modern large language models. It emphasizes key aspects such as modeling generality and capability, ensuring that the models can be applied across a wide range of tasks and perform robustly. The library also prioritizes training stability and efficiency, crucial for developing and managing large-scale foundation models. By providing tools and frameworks within the PyTorch ecosystem, torchscale aims to empower researchers and developers to build, train, and deploy increasingly complex and powerful AI models more effectively.

UCR_Time_Series_Classification_Deep_Learning_Baseline is an open-source repository designed to provide a foundational deep learning model for time series classification. It specifically utilizes fully convolutional neural networks (FCNs) to establish a robust baseline for research and application. The tool is tailored for univariate time series data, making it suitable for a wide array of domains including finance, industrial applications, and healthcare, where time-dependent data analysis is crucial. It supports both representation learning and classification tasks, offering a valuable resource for data scientists and researchers looking to explore or implement deep learning solutions for time series analysis.

UER-py (Universal Encoder Representations) is an open-source framework designed for pre-training on general-domain corpora and fine-tuning on downstream NLP tasks using PyTorch. It emphasizes model modularity, allowing users to combine various embedding, encoder, decoder, and target modules to construct custom pre-training models. The toolkit supports CPU, single GPU, and distributed training modes, making it versatile for different computational environments. UER-py also provides a comprehensive model zoo with pre-trained models of diverse properties, facilitating their direct use in various applications. It has been tested for reproducibility against original implementations of models like BERT, GPT-2, ELMo, and T5, and offers solutions for numerous NLP competitions.

ClipBERT is an official PyTorch code implementation for an efficient framework designed for end-to-end learning across image-text and video-text tasks. Recognized with a CVPR 2021 Best Student Paper Honorable Mention, ClipBERT processes raw videos/images and text inputs to generate task predictions. It leverages 2D CNNs and transformers, incorporating a sparse sampling strategy to enable efficient multimodal learning. The framework supports end-to-end pretraining and finetuning for tasks such as image-text pretraining on COCO and VG captions, text-to-video retrieval on MSRVTT, DiDeMo, and ActivityNet Captions, video-QA on TGIF-QA and MSRVTT-QA, and image-QA on VQA 2.0. Its modular design allows for easy integration of additional image-text or video-text tasks.

Leash Bio is revolutionizing drug design by building a massive, proprietary dataset of protein-molecule interactions. The platform screens millions of compounds against thousands of proteins, generating over 30 billion data points. This extensive dataset is ideal for training advanced machine learning models, enabling faster and more effective drug discovery. Leash Bio employs a dynamic, cyclical engine that continuously harnesses data, iterates machine learning, and refines its approach, with each cycle taking only a few months. Their innovative software designs and refines novel chemical matter, leading to molecules with desired activities. The company is developing internal oncology programs and partnering with biopharma companies to explore new molecule opportunities.

ultrasound-nerve-segmentation is a deep learning tutorial designed for the Kaggle Ultrasound Nerve Segmentation competition, utilizing the Keras library. This project demonstrates how to build a deep neural network for segmenting nerves in ultrasound images. The architecture is inspired by U-Net, featuring skip connections from encoder to decoder layers. It includes scripts for data processing, model definition, training, and generating submission files. The tutorial details the use of a convolutional auto-encoder, a custom Dice coefficient loss function, and Adam optimizer for training. It serves as a practical guide for those looking to implement image segmentation with deep learning, providing a foundational model that achieves a competitive score on the leaderboard.

MedSegDiff is an open-source framework that leverages Diffusion Probabilistic Models (DPM) for the segmentation and reconstruction of organs and tissues from medical images. It offers two versions, MedSegDiff-V1 and MedSegDiff-V2, with the latter incorporating Transformers for improved accuracy and stability. The tool provides scripts for training and sampling on datasets like ISIC for melanoma segmentation and BRATS2020 for brain tumor segmentation. It supports multi-GPU distributed training and includes DPM-Solver for faster sampling. MedSegDiff is designed for researchers and developers in medical image analysis, offering flexibility to run on custom datasets by implementing new data loaders.

dm_control is Google DeepMind's comprehensive software stack designed for physics-based simulation and Reinforcement Learning (RL) environments, built upon the MuJoCo physics engine. It offers Python bindings to the MuJoCo engine, a suite of RL environments, and an interactive viewer for real-time interaction. The package also includes libraries for composing and modifying MuJoCo MJCF models in Python, defining rich RL environments from reusable components, and additional libraries for custom tasks like multi-agent soccer. This open-source tool is ideal for researchers and developers working on advanced AI and robotics applications, providing a robust infrastructure for developing and testing continuous control algorithms.

nn_vis is an open-source project designed for processing and rendering neural networks to visualize their architecture and parameters. Developed as part of a master's thesis, it introduces a novel 3D visualization technique that declutters complex models. The tool estimates attributes for trained neural networks using established optimization methods like batch normalization, fine-tuning, and feature extraction to determine the importance of different network parts. It combines these importance values with techniques such as edge bundling, ray tracing, 3D impostors, and special transparency to create a comprehensive 3D model. nn_vis supports both 2D and VR visualization, allowing users to gain insights into model behavior, especially regarding generalization based on edge proximity. It also provides a GUI for controlling shader parameters and processing settings, enabling customization of the visualization.

Federated-Learning-PyTorch provides an open-source implementation of the vanilla federated learning paradigm, as described in the paper 'Communication-Efficient Learning of Deep Networks from Decentralized Data'. This tool is built using PyTorch and allows researchers and developers to conduct experiments on popular datasets such as MNIST, Fashion MNIST, and CIFAR10. It supports both independent and identically distributed (IID) and non-IID data distributions, with options for equal or unequal data splits among users. The implementation focuses on simple models like MLP and CNN to illustrate the effectiveness of federated learning, making it a valuable resource for understanding and experimenting with this distributed machine learning approach.

facial-expression-recognition-using-cnn is an open-source project designed for deep facial expression recognition using Convolutional Neural Networks (CNN) with OpenCV and TensorFlow. It can analyze facial expressions from both static images and real-time camera streams, categorizing them into emotions like Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral. The tool allows for training models on datasets like Fer2013, optimizing hyperparameters, and evaluating performance. It supports the integration of additional features such as face landmarks and HOG features to improve accuracy, providing a robust framework for researchers and developers interested in emotion detection and facial analysis.

fairchem is a comprehensive, open-source library developed by the FAIR Chemistry team, offering machine learning methods specifically tailored for chemistry. It serves as a centralized repository for data, models, demos, and applications in materials science and quantum chemistry. The library supports various tasks, including relaxing adsorbates on catalytic surfaces, optimizing inorganic crystals, running molecular dynamics simulations, and calculating spin gaps. It features pretrained models like UMA, which can be used with the ASE FAIRChemCalculator for a wide range of applications. fairchem also supports multi-GPU inference and LAMMPs integration for large-scale simulations, making it suitable for complex computational chemistry problems.