Sonar
vLLM-based LLM inference engine — formerly known as Aphrodite Engine.
About
Sonar is an open-source LLM inference engine built on vLLM, developed by the PygmalionAI/Ruliad community. It provides an OpenAI-compatible API server with quantization support, speculative decoding, multimodal model support, multi-LoRA serving, and distributed inference options. It grew out of the roleplay and open-model community's need for high-throughput, easy-to-deploy text generation servers.
The important context: the project was renamed from Aphrodite Engine to Sonar — older guides, Docker images, and configs referencing aphrodite-engine are stale. As a vLLM derivative, it inherits vLLM's strengths and its hardware assumptions (CUDA-first). Evaluate it against upstream vLLM and other community servers on your specific workload; the differentiator is its community-driven feature set rather than raw throughput leadership.
What it can do
- OpenAI-compatible inference server
- Quantization support
- Speculative decoding
- Multimodal model serving
- Multi-LoRA adapter serving
Setup
Install via the Sonar repository (formerly aphrodite-engine) and launch the OpenAI-compatible server.
Prerequisites
CUDA-capable GPU; Python environment with vLLM dependencies.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 85; Trending score 70/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.