PrivateGPT
The open-source API layer that turns local models into production AI apps — RAG, tools, MCP, no cloud.
About
PrivateGPT is the open-source API layer for building private AI applications on local models. Running a model locally is only the first step — PrivateGPT provides the higher-level building blocks as a Claude-API-style interface: a standard messages API with streaming and token counting, file and artifact ingestion, retrieval with citations and agentic RAG, built-in tools (web search, web fetch, code execution), custom tools, MCP connectors, structured access to databases and CSVs, embeddings, and orchestration.
It does not run models itself — it connects to any OpenAI-compatible inference server (Ollama, llama.cpp, vLLM) via OPENAI_API_BASE, and ships a built-in workbench UI at /ui for testing. With ~57,600 GitHub stars it is one of the most-starred local-AI projects ever, and version 1.0 (June 2026) merged two years of Zylon's private-fork work back into the open repo, turning it from a semantic-search demo into a full application backend. Zylon sells the enterprise platform built on top.
The honest caveat: 1.0 was a breaking release — the original PrivateGPT API is not forward-compatible, so older tutorials and integrations need the migration guide. And 'private' still means you operate the inference server yourself; the privacy win comes with ops responsibility.
What it can do
- Claude-API-style messages API: streaming, async, token counting
- File/artifact ingestion with retrieval, citations, and agentic RAG
- Built-in tools mirroring the Claude API: web search, web fetch, code execution
- Custom tools, MCP connectors, skills, and structured database/CSV access
- Works with any OpenAI-compatible server (Ollama, llama.cpp, vLLM)
- Built-in workbench UI at /ui; Apache-2.0 licensed; ~57.6k GitHub stars
Setup
- Start an OpenAI-compatible inference server (e.g.
ollama servewith a pulled LLM + embedding model). - Install:
brew install private-gpt(macOS, via zylon-ai/tap) oruv tool install 'private-gpt[core]'(Linux/Windows). - Run:
OPENAI_API_BASE=http://localhost:11434/v1 private-gpt serve. - Open http://localhost:8080/ui to test, or build against the API at http://localhost:8080.
Prerequisites
A running OpenAI-compatible LLM server (Ollama recommended) plus an embedding model; Python 3.11 for the uv install path.
Use cases
Works with
Trust score — 9/10
Scored 9/10 (verified tier, metrics-v1). Evidence: Adoption 9: starScore from 57,582 stars on zylon-ai/private-gpt — one of the most-starred local-AI projects; Maintenance 9: pushed 2026-10-15 (recency 10); 1.0 + 1.0.1 releases in 2026 assumed 6; issues ratio 4/57582 -> 10; Manual 9: 1.0 release notes + README verified; deduction: 1.0 broke API compat — migration caveat flagged, not hidden. License: Apache-2.0.
★ 57,582 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 9.0/10
- Maintenance · recency, releases, issue load
- 9.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 9.0/10
- · Adoption 9: starScore from 57,582 stars on zylon-ai/private-gpt — one of the most-starred local-AI projects
- · Maintenance 9: pushed 2026-10-15 (recency 10); 1.0 + 1.0.1 releases in 2026 assumed 6; issues ratio 4/57582 -> 10
- · Manual 9: 1.0 release notes + README verified; deduction: 1.0 broke API compat — migration caveat flagged, not hidden
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.