Opik

Opik is Comet's open-source platform for the full LLM application lifecycle: trace every LLM call with full trace trees for multi-step agents and tool calls, run evaluations with datasets and LLM-as-a-judge metrics (hallucination detection, moderation, RAG assessment), manage prompts in a playground, and monitor production with dashboards and online evaluation rules. A PyTest integration lets you test LLM pipelines on every commit, and the Agent Optimizer SDK tunes prompts and agents automatically.

It integrates natively with the frameworks teams actually use — Google ADK, AutoGen, Flowise, LangChain, and more — and it is Apache-2.0 with a genuinely self-hostable full platform (Docker Compose), not just an open client. At 22,600+ stars it is the leading open alternative to LangSmith-style tracing. The business model is the usual open-core shape: the platform is free to self-host, and Comet's cloud is where the upsell lives, so evaluate the managed tier on its own merits.

1. Self-host with Docker Compose, or sign up for Opik Cloud (free tier).
2. Install the SDK: pip install opik (TypeScript SDK also available).
3. Initialize in your code and wrap LLM calls — traces appear in the dashboard.
4. Build datasets and run experiments with LLM-as-a-judge metrics.
5. Add the PyTest integration to CI and set production monitoring rules.