OM1, developed by OpenMind, is a modular AI runtime designed to empower developers in creating and deploying multimodal AI agents for both digital environments and physical robots. This includes humanoids, phone apps, quadrupeds, and educational robots like TurtleBot 4, as well as simulators such as Gazebo and Isaac Sim. OM1 agents can process a wide range of inputs, including web data, social media, camera feeds, and LIDAR, and facilitate physical actions like motion, autonomous navigation, and natural conversations. The platform emphasizes ease of upgrade and reconfiguration to accommodate various physical form factors. Key features include a Python-based modular architecture, easy handling of new data and sensors, hardware support via plugins for ROS2, Zenoh, and CycloneDDS, and a web-based debugging display (WebSim). It also offers pre-configured endpoints for multiple LLMs (OpenAI, xAI, DeepSeek, Anthropic, Meta, Gemini, NearAI, Ollama) and Visual Language Models (VLMs).
Best used for
Ideal for developers and data scientists who need to create and deploy advanced multimodal AI agents for robotics and digital environments. Especially valuable for those looking to integrate diverse data inputs, enable complex physical actions, and leverage pre-configured LLM/VLM endpoints for rapid prototyping and deployment on various robotic platforms.
Common actions