Infinity
MIT-licensed embedding and reranker inference with dynamic batching.
About
Infinity is a MIT-licensed inference server for embedding and reranker models with dynamic batching — a community alternative to Hugging Face's TEI. 3k stars, actively maintained. It handles the same job (serve bge/e5/jina-class models behind an API) with a permissive license and a simpler codebase to hack on. Smaller community and adoption than TEI, but the MIT license and active development make it worth knowing for embedding infrastructure.
What it can do
- **Dynamic batching**: efficient GPU utilization
- **Embeddings + rerankers**: both serving modes
- **MIT licensed**: permissive, hackable codebase
- **Multiple engines**: torch, optimum, CTranslate2 backends
Setup
```bash
pip install infinity-emb
infinity_emb v2 --model-id BAAI/bge-small-en-v1.5 --port 8080
```
Docs in the repo.
Prerequisites
Python 3.9+; GPU recommended for larger models.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (verified tier, metrics-v1). Evidence: 2,956 GitHub stars (log-scaled vs catalog peers); blended with ~4,797 monthly package downloads; last push 3d ago; 0 releases/12mo; 141 open issues. Manual review: MIT embedding/reranker inference (dynamic batching), 3k stars, active; small community vs TEI; modest PyPI traction. License: MIT.
★ 2,956 GitHub stars
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.4/10
- Maintenance · recency, releases, issue load
- 6.1/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · 2,956 GitHub stars (log-scaled vs catalog peers)
- · blended with ~4,797 monthly package downloads
- · last push 3d ago; 0 releases/12mo; 141 open issues
- · manual review: MIT embedding/reranker inference (dynamic batching), 3k stars, active; small community vs TEI; modest PyPI traction
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.