Shypd.aiThe largest open index of AI toolsLlamaEdge is an AI Agents & Automation tool that provides the easiest and fastest way to run customized and fine-tuned LLMs locally or on the edge. It allows users to create OpenAI-compatible API services for open-source LLMs.
Trending
About
LlamaEdge is an open-source project designed to simplify the deployment and execution of Large Language Models (LLMs) locally or on edge devices. It enables users to run customized and fine-tuned LLMs with ease and speed, offering a robust solution for local inference. A key feature is its ability to create OpenAI-compatible API services for various open-source LLMs, supporting text generation, embeddings, speech-to-text, text-to-speech, and text-to-image models. Built on a Rust+Wasm stack, LlamaEdge offers a lightweight, fast, portable, and secure environment for AI inference, compatible with multiple operating systems, CPUs, and GPUs. It supports GGUF-formatted LLMs based on the Llama2 framework and provides a command-line interface for interaction.
Capabilities
Pricing & Plans
Open Source
Free
FAQs
Trending
