Llama-Gpt

Visit Tool

llama-gpt is a self-hosted, offline, ChatGPT-like chatbot powered by Llama 2 and Code Llama. It offers 100% private conversations with no data leaving your device.

Claim this tool

No Views Yet

At a glance

Pricing

Open Source

Free tier

Yes

API

Yes

Skill level

Technical

About

What is llama-gpt?

llama-gpt is a self-hosted, offline, ChatGPT-like chatbot that prioritizes privacy and local processing. Powered by Llama 2 and Code Llama models, it ensures that no data leaves your device, offering a secure and private conversational AI experience. It supports various Llama 2 and Code Llama models, including 7B, 13B, 34B, and 70B sizes, and can be installed on umbrelOS home servers, M1/M2 Macs, or any system with Docker. The tool also features an OpenAI-compatible API for easy integration and supports Nvidia GPUs for enhanced performance. Benchmarks are provided for different hardware configurations, showcasing its generation speed.

Best used for

Ideal for developers and privacy-conscious individuals who need to run a private, offline ChatGPT-like chatbot, generate code locally, and ensure no data leaves their device. Especially valuable for those requiring an OpenAI-compatible API for custom integrations and leveraging Nvidia GPUs for performance.

Common actions

deploy AI chatbot

generate code

process data privately

run LLMs locally

"AI Agents"github copilotface swappingautomated workflowdeepfakecollaborationopen-sourceworkflowslow-code/no-code

Capabilities

Key features

Self-hosted chatbot
Offline operation
Llama 2 support
Code Llama support
Nvidia GPU support
OpenAI-compatible API

Target Audience

developersprivacy-conscious usersresearchersit professionals

Integrations

Not yet documented

Pricing & Plans

Open Source

Free

FAQs

What models does llama-gpt support?

llama-gpt supports various Llama 2 Chat models (7B, 13B, 70B) and Code Llama Chat models (7B, 13B, 34B). It also supports Phind Code Llama 34B Chat. Support for running custom models is on the roadmap.

Can I run llama-gpt on my local machine?

Yes, llama-gpt can be installed on umbrelOS home servers, M1/M2 Macs, and any x86 or arm64 system with Docker. Installation instructions are provided for each platform, including options for Nvidia GPU support.

Does llama-gpt offer an API?

Yes, llama-gpt provides an OpenAI-compatible API, thanks to llama-cpp-python. This allows for easy integration with other applications and tools, with API documentation available at http://localhost:3001/docs.

Trending

Subcategories trending in Customer Support & CX

Voice & Call Center Email & Chat Support Customer Feedback Community Management Customer Success Multilingual Support

Trending

Explore

Browse AI tools by category

Content & Design Productivity & Business Coding & Development AI Agents & Automation Research & Education Wellness & Lifestyle Career Development Marketing & Growth Data & Analytics Customer Support & CX Finance E-commerce