Coding & Development

LLM-Viewer is a comprehensive tool designed for visualizing and analyzing the performance of Large Language Models (LLMs) across various hardware platforms. It provides in-depth network-wise analysis, allowing users to understand critical factors such as peak memory consumption and total inference time cost. The tool supports both a user-friendly web interface for easy configuration and visualization, and a command-line interface (CLI) for more programmatic use. LLM-Viewer helps users gain valuable insights into LLM inference and optimize performance by considering computation, storage, transmission, and hardware roofline models. It's an ongoing project with plans for expanded hardware and LLM compatibility.

Loglizer is an open-source machine learning toolkit designed for automated anomaly detection and intelligent fault diagnosis based on system logs. It provides a comprehensive framework that includes log collection, log parsing (leveraging the logparser project), feature extraction, and anomaly detection. The toolkit implements various machine learning models, including both supervised (LR, Decision Tree, SVM) and unsupervised (LOF, One-Class SVM, Isolation Forest, PCA, Invariants Mining, Clustering) approaches. Loglizer is particularly useful for developers and data scientists who need to monitor system operations, track abnormal behaviors, and proactively identify errors within complex software systems. It offers a practical solution for analyzing large volumes of log data to ensure system reliability and performance.

LLM Gateway is an open-source API gateway designed to streamline the management and analysis of Large Language Model (LLM) requests. It acts as a middleware between applications and various LLM providers, including OpenAI, Anthropic, and Google Vertex AI. Key functionalities include routing requests to different providers, centralizing API key management, and tracking token usage and costs. The platform also provides performance monitoring and usage analytics to help users optimize their LLM interactions. It offers a unified API interface compatible with the OpenAI API format for seamless integration and supports both hosted and self-hosted deployment options.

MORSE Corp is an employee-owned company specializing in algorithm and software development services. Their team comprises talented engineers, software developers, and scientists with diverse technical capabilities, rooted in Aerospace Engineering. They apply a user-centered development process, continuously engaging with users to understand needs and gather feedback. A key differentiator is their hands-on approach to field testing alongside users in operationally representative conditions, ensuring products are tailored to user needs and operational constraints. This approach avoids forcing users to adapt their tactics or procedures to suit the product, instead delivering solutions that seamlessly integrate into existing workflows.

GrapixAi is an innovative AI tool that specializes in creating unique and engaging online game experiences. The platform focuses on generating AI-driven content within games that aims to evoke a humorous mix of frustration and addiction among players, leading to viral trends and discussions. It's designed to make online gaming more dynamic and entertaining by introducing elements that are perceived as annoying yet ultimately captivating. The tool's impact is highlighted by numerous articles on its website, suggesting it transforms traditional online games into platforms for "funny anger" and widespread player engagement, particularly within the Indonesian gaming community.

Paddler is an open-source load balancer and serving platform designed for self-hosting Large Language Models (LLMs) and Vision Language Models (VLMs) at scale. It offers a streamlined alternative to existing solutions like llm-d or Docker Model Runner, focusing on fewer moving parts and simpler deployments built around the ggml ecosystem. Key features include a built-in llama.cpp engine for inference, LLM-specific load balancing, dynamic model swapping, and request buffering for scaling from zero hosts. Paddler also provides a web admin panel for management, monitoring, and testing, along with observability metrics. It runs efficiently on both CPU and GPU, catering to product teams needing LLM inference, DevOps/LLMOps teams deploying models at scale, and organizations with high compliance and privacy requirements.

Bitdeer AI offers a comprehensive AI cloud platform designed for building, training, and deploying AI applications. Leveraging NVIDIA GPUs, including the NVIDIA GB200 NVL72 Superchip, it provides scalable compute resources and intelligent AI Cloud services for production workloads. The platform includes GPU Cloud Services like Virtual Machines, Bare Metal, and Container Services, alongside AI Studio & AI Solutions for serverless models, distributed training, and an AI Agent Platform. Users can train AI models faster with advanced NVIDIA accelerated computing infrastructure, deploy models quickly and reliably, and run real-time AI at scale with serverless API endpoints. It supports end-to-end AI workloads by combining IaaS, PaaS, SaaS, and MaaS, facilitating efficient AI development and deployment through AI Studio, AI Agent Builder, model libraries, and managed databases.

SqueezeLLM is a post-training quantization framework designed to optimize the serving of large language models (LLMs) through a novel Dense-and-Sparse Quantization method. This approach addresses the significant memory requirements of LLMs by splitting weight matrices into a dense component, which can be heavily quantized without performance loss, and a sparse component that preserves sensitive outlier parts. This allows for serving larger models with a smaller memory footprint, maintaining the same latency, and achieving higher accuracy and quality compared to baseline models. For instance, SqueezeLLM's variant of Vicuna models can operate within 6 GB of memory, surpassing FP16 baseline models in MMLU accuracy despite the latter requiring twice the memory. The framework supports various LLMs including LLaMA, LLaMA-2, Mistral, Vicuna, XGen, and OPT, with options for 3-bit and 4-bit quantization and different sparsity levels.

Stable-Diffusion-WebUI-TensorRT is a TensorRT extension designed to significantly boost the performance of Stable Diffusion Web UI on NVIDIA RTX GPUs. This tool is compatible with a range of Stable Diffusion models, including 1.5, 2.1, SDXL, SDXL Turbo, and LCM, ensuring broad applicability for users. To leverage its capabilities, users must install the extension and generate optimized engines, following the detailed instructions provided. This optimization is crucial for achieving faster inference times and a smoother workflow when generating images, making it an essential addition for developers and graphic designers working with Stable Diffusion on NVIDIA hardware.

stoolap, powered by GitHub, offers a comprehensive platform for software development, AI code creation, and application security. It provides tools like GitHub Copilot for writing better code with AI, GitHub Spark for building intelligent apps, and GitHub Models for managing prompts. Developers can automate workflows with GitHub Actions, use instant dev environments with Codespaces, and manage code changes with robust code review features. The platform also includes advanced security features like GitHub Advanced Security to find and fix vulnerabilities, and secret protection to prevent leaks. It caters to individuals, teams, and enterprises, offering various plans with features for collaboration, project management, and deployment.

Integration Wizards Solutions is an AI-powered company specializing in computer vision and enterprise mobility products. Their flagship product, IRIS, leverages existing CCTV infrastructure to provide actionable intelligence from live data. IRIS offers solutions for retail, analyzing customer behavior, staff utilization, and store layout. In petroleum retail, it identifies customers and vehicle analytics, optimizing standard operating procedures. For health & safety, IRIS detects non-compliance, such as fire or PPE issues, and generates daily reports on machine utilization. It also enhances security by converting CCTV cameras into dependable security solutions that detect human intrusion. Additionally, Integration Wizards provides Silverline, an enterprise mobility platform, and Silverline MDM for remote mobile device management.

tensor_parallel is a Python library designed to automatically split PyTorch models across multiple GPUs, facilitating both training and inference for large language models (LLMs). This tool allows users to run models that would otherwise exceed the memory capacity of a single GPU, offering potentially linear speedups. It simplifies the process with a single line of code integration and supports memory-efficient dispatch by converting state_dicts. Key features include options for custom parallelism strategies, distributed training with `torch.distributed`, and sharding parameters using the ZeRO-3 algorithm to avoid duplicate parameters. It is particularly useful for quick prototyping on a single machine with multiple GPUs, offering an easier setup compared to more complex distributed training frameworks.

Apache TVM is an open machine learning compilation framework designed for Python-first development, allowing for quick customization of machine learning compiler pipelines. It focuses on universal deployment, enabling models to be integrated into minimum deployable modules. The project has evolved significantly, now featuring TensorIR as a tensor-level representation and Relax as a graph-level representation, with a strong emphasis on Python-first transformations. This design makes ML compilers more accessible by allowing most transformations to be customizable in Python, optimizing computational graphs, tensor programs, and libraries. TVM also serves as a foundational infrastructure for building Python-first vertical compilers, particularly for domains like Large Language Models (LLMs).

Radiant, formerly known as Ori, is an integrated AI infrastructure and cloud platform designed to power the AI era. It unifies software, power, land, capital, and compute into a vertically integrated system, defining a utility model for AI. The platform offers a complete AI Cloud featuring NVIDIA accelerated computing and MLOps capabilities, including Inference, Fine-Tuning, Model Registry, Serverless Kubernetes, and high-performance storage. Radiant's proprietary software platform is built on engineering first principles, ensuring intelligent scheduling, automated node management, secure multi-tenancy, and a distributed control panel. As an NVIDIA Cloud Partner, Radiant engineers and operates its AI Factories using the latest NVIDIA GPU architectures. With access to over 5 GW live and 45 GW of renewable generation capacity globally, Radiant operates the world’s largest powered-land portfolio, enabling rapid deployment of massive AI compute. Backed by Brookfield with over $100 billion in deployable capital, Radiant is positioned to undertake large-scale projects in the AI ecosystem.

Outspeed builds tooling and infrastructure to power lifelike and emotive AI companions, enabling human-like voice interaction through its SDK and API. The platform is designed for high concurrency, ensuring reliable performance for all users. Key features include natural prosody and emotion in generated voices, ultra-low latency for smooth conversations, and easy integration with simple APIs and clear documentation. It supports multilingual capabilities and offers scalable infrastructure, making it suitable for developers looking to add advanced voice features to their AI applications.

Rain AI is focused on developing highly energy-efficient hardware specifically designed for artificial intelligence. The company's core mission is to create a compute platform that will power the future of AI infrastructure, significantly reducing the cost of AI computation. By co-designing every layer of the AI stack, from circuits to algorithms, Rain AI seeks to optimize performance and efficiency. This approach positions Rain AI as a key player in advancing sustainable and cost-effective AI development, supported by leaders in AI and venture capital firms.

Frugal is an Application Cost Engineering (ACE) platform designed to automatically reduce cloud spend by identifying and rectifying inefficient code patterns within applications. Unlike traditional FinOps tools that focus on infrastructure spend, Frugal analyzes actual source code to pinpoint and resolve costly application behaviors, such as excessive metrics reporting or inefficient AI model calls. Its AI agents generate pull requests to reduce the consumption of high-cost cloud services without hindering developer productivity. After optimizing production applications, Frugal integrates into CI/CD pipelines to prevent future waste, effectively bridging the gap between FinOps and engineering teams. It works across various coding languages and major cloud platforms like AWS, GCP, Azure, and third-party services.

Humanoid is at the forefront of developing advanced modular humanoid robots, focusing on commercial scalability and safety. Their robots, including the HMND 01 Alpha Wheeled and Alpha Bipedal models, are powered by Humanoid’s VLM and VLA-based KinetIQ framework. This AI framework enables efficient orchestration of robot fleets across various platforms, designed for field-tested performance in industrial settings and humanlike adaptability. The company aims to integrate these robots into diverse industries, addressing needs for automation and advanced robotic capabilities. Humanoid is actively building a team to push the boundaries of robotics and AI, partnering with global companies to accelerate development and deployment.

Grahamai is a blog dedicated to exploring the multifaceted world of tech influence, with a particular emphasis on the role of artificial intelligence. It features articles covering a range of topics, from the ethical considerations and potential pitfalls of tech influence to practical lessons for entrepreneurs derived from top tech influencers. The platform also delves into how AI is being utilized by influencers to create better content and how these individuals are shaping the future of AI and innovation. Readers can find insights on evaluating the impact of tech influencers, understanding their journey from content creator to thought leader, and identifying key traits of truly genius tech influencers.

CloudTruth is a Config Data Platform designed to eliminate misconfigurations and accelerate software delivery by centralizing secrets and configuration data management. It helps organizations reduce outage hours, mitigate security risks, and increase operational capacity by automating manual and repetitive tasks. The platform offers features like Config Secrets Copilot, scheduled secret rotations, centralized compliance, and accurate configuration for every deployment. CloudTruth integrates with popular tools like Terraform, Kubernetes, GitHub, AWS Secret Manager, and Azure Key Vault, making it suitable for software developers, QA, SRE, DevOps, Platform Engineers, and CISOs involved in building, shipping, operating, and securing applications.

Preventio leverages cutting-edge AI technology to revolutionize the monitoring and maintenance of pipeline networks. By analyzing historical and current data, the platform accurately detects leaks and anomalies within utility networks, addressing challenges posed by aging infrastructure and rising costs. Preventio offers three core solutions: Leak Detection & Localization, which uses AI to pinpoint leaks; Predictive Maintenance, optimizing maintenance strategies in water, district heating, and industrial sectors to proactively save resources and costs; and Risk Scoring, an innovative solution that uses AI and advanced analytics to assess and manage infrastructure risks for insurers, housing associations, and utility companies. The tool aims to enhance efficiency and reliability, making infrastructure damages a thing of the past.

up2metric is an SME specializing in innovative custom software solutions and consulting engineering services across Computer Vision, Machine Learning, Photogrammetry, Remote Sensing, and Metrology. They assist partners from initial AI world steps, designing products, gathering image data, developing applications, and deploying them to dedicated hardware, portable devices, or mobile phones. The company offers end-to-end 3D reconstruction services for complex objects, from small artifacts to large geographic regions, and specializes in integrating multiple sensors for multi-modal data analytics. up2metric also engages in research projects, aiming to transfer state-of-the-art knowledge from academia to the market. Their services include Computer Vision AI, Custom Software Development, 3D Services, Visual Data Analysis, and Research.

Whitespace offers Collective, an AI operating system specifically built for regulated and security-first industries. It enables organizations to deploy AI in challenging environments such as air-gapped systems, at the edge, and in the field. Collective features a modular architecture with an intuitive app-style interface, allowing users to access and launch AI applications for chat, lessons, insights, and decisions. The platform is designed for scale, trust, and action, supporting secure, high-value AI across sectors like Defence, Government, Financial Services, Emergency Services, and Healthcare. It integrates cleanly with existing systems and provides full visibility and auditability for AI governance, making it suitable for CTOs, CIOs, Heads of AI/ML, Chief Risk Officers, and Transformation Leaders.

NeuReality is dedicated to transforming AI infrastructure by addressing system bottlenecks and enhancing GPU utilization. Their core offerings include NR-NEXUS, an Inference OS for Token Factories that orchestrates models, runtimes, and workloads across various cloud and XPU infrastructures, and NR2 AI-SuperNIC, networking silicon designed to eliminate data-movement bottlenecks in large-scale AI environments. These solutions aim to improve infrastructure cost and energy efficiency, ultimately converting AI into practical business value. NeuReality also provides an AI-CPU engineered for inference at scale and an AI-Inference Appliance that doubles average GPU utilization, making AI adoption faster and more impactful for businesses.