Amphion is an MIT-licensed Python toolkit for audio, music and speech generation that gives researchers and engineers a shared, reproducible base for tasks such as text to speech, singing voice synthesis, voice conversion and text to audio.

What it is

Amphion lives in the open-source Python audio and machine-learning research ecosystem, maintained under the open-mmlab GitHub organisation. The repository states a North-Star objective of offering a platform for studying the conversion of any inputs into audio. It covers several individual generation tasks in one place: Text to Speech (TTS), Singing Voice Synthesis (SVS), Voice Conversion (VC), Accent Conversion (AC), Singing Voice Conversion (SVC) and Text to Audio (TTA) are supported, while Text to Music (TTM) is still in development. Beyond the tasks themselves, the project ships vocoders, which produce the high-quality audio signal, and evaluation metrics, which keep scores comparable across experiments. It also includes work on building large-scale datasets for speech synthesis, such as Emilia-Large at over 200,000 hours, and it offers visualisations of classic models and architectures, which the README presents as its unique feature.

The concrete problem it addresses is fragmentation. Each generation task normally arrives as its own paper, its own codebase, its own vocoder choice and its own measurement convention, so results are hard to compare and hard for a newcomer to enter. Amphion gathers the tasks, the vocoders and the evaluation metrics behind one project, which is what allows the metrics to stay consistent between generation tasks and what supports reproducible research. The stated audience is junior researchers and engineers who want to get started in audio, music and speech generation research and development, and the architecture visualisations exist specifically so that those readers can understand a model before running it. Distribution follows that audience: code on GitHub, pretrained checkpoints on Hugging Face, and at least one model runnable from a Colab notebook.

Key capabilities

• Six supported generation tasks in one toolkit: TTS, SVS, VC, AC, SVC and TTA, with TTM in development.
• Vevo2 (models/svc/vevo2/README.md), a unified and controllable framework that bridges speech and singing voice generation through unified prosody learning, covering TTS, SVS, VC, SVC, singing voice editing, singing style conversion and melody control.
• DualCodec (models/codec/dualcodec/README.md), a low-frame-rate neural audio codec operating at 12.5Hz or 25Hz, semantically enhanced with an SSL feature, extracting discrete tokens for efficient speech generation.
• Metis (models/tts/metis), a foundation model for unified speech generation supporting zero-shot text-to-speech, voice conversion, target speaker extraction, speech enhancement and lip-to-speech.
• Vevo1.5 (models/svc/vevosing/README.md), which extends Vevo for unified and controllable generation of both speech and singing voice.
• Bundled vocoders and evaluation metrics, so audio quality and scoring stay consistent across tasks.
• Large-scale dataset work, including the Emilia-Large dataset with over 200,000 hours of data.

Who uses it and how

• Junior researchers and engineers entering audio, music and speech generation research and development, using the architecture visualisations as the entry point.
• Research groups that need reproducible experiments and consistent evaluation metrics across several generation tasks rather than one metric per paper.
• Teams building speech-synthesis datasets at scale, using the project's large-scale dataset work such as Emilia-Large.
• Singing voice and voice conversion practitioners, who can work with Vevo2 and Vevo1.5 for melody control and singing style conversion.
• Developers needing zero-shot text-to-speech, speech enhancement or lip-to-speech, who can adopt Metis as a foundation model.

Getting started

Clone the open-mmlab/Amphion repository and follow the per-model README files, such as models/tts/metis, models/svc/vevo2/README.md, models/svc/vevosing/README.md and models/codec/dualcodec/README.md, with pretrained checkpoints published on Hugging Face as amphion/metis and RMSnow/Vevo2. DualCodec is additionally runnable from a hosted Colab notebook, and Vevo2 has a demo page at versasinger.github.io.

How it compares

No paid products or comparable third-party toolkits are named in the facts for this page. The model names that appear as topics on the project, such as FastSpeech2, NaturalSpeech2, MaskGCT and AudioLDM, are tags on Amphion rather than separate alternatives to it, so it stands alone in this registry entry.

When to use it — and when not to

A self-hoster must operate a Python environment and work through each model's own README, and training or fine-tuning models such as Metis or Vevo2 means supplying the compute and managing large datasets such as Emilia-Large independently; no pip package, Docker image or compose file is given in the facts, so there is no single-command deployment path. Teams that want a turnkey hosted API, a support contract or a managed service should not pick this project, and Text to Music is not yet supported. The repository also carries 174 open issues, its registry category of Business Software / Compliance & Risk Management does not describe an audio generation toolkit, and the tagline is truncated in the listing, so readers should go to the project documentation rather than rely on this page alone.