Skip to content
← Models
Open weightsAlibaba DAMO Academy/10 trustVerified

3D-Speaker

Alibaba's multi-modal speaker toolkit: verification, recognition and diarization (audio-only or audio+video) with challenge-grade models like CAM++ and ERes2NetV2, all Apache 2.0.

Visit official page →

About

3D-Speaker (modelscope/3D-Speaker, ICASSP 2025 toolkit paper, arXiv 2306.15354) is Alibaba DAMO Academy's open toolkit for speaker verification, recognition and diarization — notable for being one of the few open projects that does audio-visual diarization (face + voice) as well as audio-only. The model zoo is its calling card: CAM++ (7.2M params, 0.65% EER on VoxCeleb1-O), ERes2NetV2 (17.8M, 0.61% EER), ERes2Net-large (22.5M, 0.52% EER) — all competitive with the best published numbers. For diarization specifically, the repo's own benchmark table reports DER of 10.30% on Aishell-4 (vs 12.2% pyannote.audio, 11.7% DiariZen_WavLM), 11.75% on VoxConverse (vs 11.3% pyannote), 19.73% on Alimeeting (vs 24.4% pyannote), plus internal Chinese meeting sets where it leads comfortably. Those are provider-reported, but the head-to-head framing against named competitors is unusually transparent. It's also bundled with the large-scale 3D-Speaker corpus (multi-device, multi-distance, multi-dialect) for representation-disentanglement research. Everything is Apache 2.0, including models and corpus — the cleanest licensing story in this batch. Strong pick if your audio is Mandarin-heavy or you want audio-visual diarization without license anxiety.

Specs

Context window

Not applicable — utterance/segment-level speaker modeling; diarization recipes over full recordings

Pricing

Free / open source — Apache 2.0

Free quota

Fully open — GitHub/ModelScope code, pretrained models, and corpus

API access

No hosted API — self-host via GitHub/ModelScope recipes; demos on ModelScope

Weights

Open weights

Released

Not disclosed

Benchmarks

  • Speaker verification: CAM++ 0.65% EER, ERes2NetV2 0.61% EER, ERes2Net-large 0.52% EER on VoxCeleb1-O (provider-reported)\n- Diarization DER: Aishell-4 10.30% (pyannote 12.2%, DiariZen 11.7%); VoxConverse 11.75% (pyannote 11.3%); Alimeeting 19.73% (pyannote 24.4%); AMI_SDM 21.76% (provider-reported head-to-head)\n- No independent replication found

Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.

Use cases

speaker-diarizationspeaker-verificationaudio-visual-diarizationmultilingual-speechresearch

Trust score — 8/10

/10 trustVerified

Scored 8/10 (TIER verified, metrics-v1). Evidence: ICASSP 2025 peer-reviewed toolkit paper (arXiv 2306.15354); transparent head-to-head DER table against pyannote and DiariZen; maintained repo (pushed December 2025). Strengths: rare open audio-visual diarization, strong verification numbers, clean Apache 2.0 licensing across code+models+corpus, genuinely multilingual (Mandarin-strong). Docked for: (1) all benchmark figures provider-reported with no independent replication; (2) toolkit-style repo — more assembly required than a turnkey pipeline. License: Apache 2.0.

How the trust score is built

Full methodology →
Adoption · stars/downloads, log-scaled vs peers
7.0/10
Maintenance · recency, releases, issue load
8.0/10
Openness · license + docs
9.0/10
Manual review · sanity read + red-flag check
8.0/10
  • · ICASSP 2025 toolkit paper; rare open audio-visual diarization.
  • · Transparent head-to-head DER table vs pyannote and DiariZen (provider-reported).
  • · Mandarin-strong; bundled multi-device/multi-distance/multi-dialect corpus.

Sources