AudioGenie
Multi-agent framework for generating audio from multimodal inputs.
About
AudioGenie is an open research framework that tackles multimodality-to-multiaudio generation: synthesizing sound effects, speech, music, and songs from mixed text, image, and video inputs. It uses a training-free dual-layer multi-agent architecture: a generation team handles fine-grained task decomposition with an adaptive mixture-of-experts for dynamic model selection, while a supervisor team checks temporal-spatial consistency and self-corrects outputs through trial-and-error refinement loops.
The work was accepted at ACM Multimedia 2025 and ships with MA-Bench, the first benchmark for multimodality-to-multiaudio tasks, comprising 198 annotated videos with multiple audio types. The authors report state-of-the-art or comparable results across 9 metrics in 8 tasks, with user studies backing quality, alignment, and aesthetics. As a research artifact rather than a polished product, it suits researchers and engineers building audio generation pipelines rather than end users.
What it can do
- Multi-agent audio generation pipeline
- Sound effects, speech, music, and song synthesis
- Trial-and-error self-correction loops
- MA-Bench multimodality-to-multiaudio benchmark
- Project website with audio samples
Setup
Clone the GitHub repo and follow the project README.
Prerequisites
Python environment and familiarity with audio generation models.
Use cases
Works with
Trust score — 7/10
Scored 7/10 (metrics tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 95; Trending score 66/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.