Skip to content
← Models
Open weightsHKUST (Zhen Ye et al.)/10 trustMetrics-checked

X-Codec2

Single-codebook neural speech codec that fuses semantic and acoustic tokens for LLM-based TTS.

Visit official page →

About

X-Codec2 (also written X-Codec-2.0) is the neural audio codec behind the LLaSA speech-synthesis work from HKUST, published February 2025 (arXiv 2502.04128). It fuses a semantic encoder (Wav2Vec2-BERT, pretrained on 4.5M hours of audio across 143+ languages) with an acoustic encoder, then quantizes the result with a single layer of finite scalar quantization — one 65,536-entry codebook producing 50 tokens per second at 16 kHz. The single flat token stream is deliberately designed to slot into autoregressive LLM pipelines without the parallel codebook streams that codecs like EnCodec need.

Trained on 150k hours of multilingual speech, it reports strong reconstruction quality (UTMOS 4.13, WER 2.47 on LibriSpeech test-clean) and has since been picked up as the speech tokenizer in systems like NVIDIA's Audex unified audio-text model. Checkpoints are on Hugging Face (HKUSTAudio/xcodec2-hf) and the model is supported in Hugging Face Transformers.

Specs

Context window

Not applicable — 16 kHz audio in, 50 discrete tokens per second out

Pricing

Free / open weights

Free quota

Unlimited (self-hosted)

API access

No official hosted API; self-host from Hugging Face checkpoints (HKUSTAudio/xcodec2-hf)

Weights

Open weights

Released

Not disclosed

Benchmarks

  • Provider-reported (repo README): UTMOS 4.13, WER 2.47, speaker similarity 0.82, STOI 0.92, PESQ-nb 3.05 / PESQ-wb 2.44 on LibriSpeech test-clean reconstruction
  • No independent third-party benchmarks published

Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.

Use cases

text-to-speechspeech-synthesisaudio-tokenizationvoice-cloningaudio-language-models

Trust score — 7/10

/10 trustMetrics-checked

Scored 7/10 (TIER metrics, metrics-v1). Evidence: published paper (arXiv 2502.04128) with LibriSpeech reconstruction evals; open checkpoints (HKUSTAudio/xcodec2-hf); adopted as the speech tokenizer in NVIDIA's Audex; merged into Hugging Face Transformers. Strengths: open weights, real downstream adoption, clean single-stream LLM-friendly design. Docked for: (1) reconstruction numbers are author-reported; (2) license terms not confirmed in sources checked. License: not confirmed in sources checked.

How the trust score is built

Full methodology →
Adoption · stars/downloads, log-scaled vs peers
7.0/10
Maintenance · recency, releases, issue load
7.0/10
Openness · license + docs
8.0/10
Manual review · sanity read + red-flag check
7.0/10
  • · Paper arXiv 2502.04128 (Feb 2025); single FSQ codebook, 50 tok/s @ 16kHz
  • · Open checkpoints (HKUSTAudio/xcodec2-hf); merged into HF Transformers
  • · Used as speech tokenizer in NVIDIA Audex; reconstruction evals author-reported

Sources