LocalAI

LocalAI is the free, open-source, self-hosted alternative to the OpenAI API. It exposes an OpenAI-compatible (plus Anthropic Messages and Open Responses compatible) REST API for local inference, letting you run LLMs, image generation, audio transcription (whisper), TTS, embeddings, and vision models on consumer-grade hardware with no GPU required. It supports 30+ backends (llama.cpp, vLLM, diffusers, MLX, whisper.cpp...), auto-detects your GPU, installs models from a gallery or directly from HuggingFace/Ollama/OCI URLs, and supports multi-user API-key auth, quotas, and distributed mode. MIT-licensed, ~49k stars, actively developed (v4.11.0, Oct 2026).

Docker (CPU, quickest)

```bash

docker run -ti --name local-ai -p 8080:8080 \

-v "$PWD/models:/models" \

localai/localai:latest-aio-cpu

```

Docker (NVIDIA GPU)

```bash

docker run -ti --name local-ai -p 8080:8080 --gpus all \

localai/localai:latest-gpu-nvidia-cuda-12

```

(AMD/ROCm, Jetson L4T, and -extras variants also exist.)

Run a model (CLI)

```bash

local-ai run qwen3-4b

local-ai run huggingface://TheBloke/phi-2-GGUF/phi-2.Q8_0.gguf

local-ai models install <name> # from the gallery

```

Call the API (OpenAI-compatible)

```bash

curl http://localhost:8080/v1/chat/completions \

-H "Content-Type: application/json" \

-d '{"model": "qwen3-4b", "messages": [{"role": "user", "content": "Hello!"}]}'

```