Shypd.aiThe largest open index of AI toolsVideoLLM-online is an AI research tool that provides an online video large language model for streaming video. It supports real-time interaction and proactively updates responses during a video stream.
Trending
Also listed in
About
VideoLLM-online is the official implementation of an Online Video Large Language Model for Streaming Video, presented at CVPR 2024. Unlike traditional models that process full videos offline, VideoLLM-online enables real-time interaction within a video stream, allowing it to proactively update responses based on activity changes or assist with next steps. It features a cheap and scalable method for synthesizing streaming data by transforming offline annotations into dialogue data using open-source LLMs. The inference method is parallelized, combining video encoding, LLM forwarding, and response generation asynchronously, achieving high speeds of 10-15 FPS on an A100 GPU for long-form videos up to 10 minutes. The tool is designed for researchers and developers working with streaming video analysis and real-time multimodal AI.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
