Confucius4-TTS

Confucius4-TTS

Speaks your text in 14 languages and can copy a reference voice across languages without extra training. A text-to-speech system from NetEase Youdao for developers and creators. Use only voices you have the right to use.

Made by netease-youdao on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Confucius4-TTS is a text-to-speech system from NetEase Youdao that pairs a speech encoder with a large language model. It can clone a voice from a short reference recording and keep that voice when it switches language. The card lists 14 languages and says no reference transcript is needed. Use only voices you have the right to use.

Who it suits: It suits developers and creators who want multilingual speech in one consistent voice, such as narration, dubbing or product voices.

Industry | Who uses it | How they use it | Example
Media | Video and audio creators | Narrate content in several languages with one voice | The same narrator reading a script in English and Japanese
Software | Developers | Add multilingual speech to an app | A script that turns text into a wav file
Education | Language teachers | Produce listening material in different languages | Sentences read aloud in a steady voice
Research | Speech researchers | Study zero-shot and cross-lingual synthesis | Compare voice transfer across languages

If you cannot confirm you have the right to use a voice, you can skip the voice cloning features.

Where to use it: You run it from Python, and the card lists Python 3.10 and CUDA 12.6 as requirements. It also points to an online demo on Hugging Face.

• Python 3.10 with conda
• A CUDA-capable GPU, though the card's Python example falls back to CPU
• Windows, macOS or Linux with the huggingface_hub package, for the download

Use case | Who | What they get back
Voice cloning | Creators | Speech in a reference voice, with no transcript needed
Cross-lingual voice | Dubbing teams | The same voice speaking another language
Emotion transfer | Producers | Speech that carries the feeling of the reference

How to set it up: You need Python 3.10, CUDA 12.6, conda and git, plus the project's code and the model files. The card clones its GitHub repository, creates a conda environment and installs the requirements.

• Python 3.10 and conda
• CUDA 12.6
• git, to fetch the project's repository

1. Clone the project's repository from GitHub, as the card does.
2. Create and activate a conda environment.
3. Install the dependencies.
4. Download the model files.

cd Confucius4-TTS
conda create -n confuciustts python=3.10 -y
conda activate confuciustts
pip install -r requirements.txt
pip install -U huggingface_hub
hf download netease-youdao/Confucius4-TTS

How to use it: Run the example script with a reference recording, the text, a language code, an output file and the config file. You can also call the Python API directly, as the second block shows.

python example.py \
    --prompt_wav path/to/reference.wav \
    --text "Your text to synthesize" \
    --lang en \
    --out output.wav \
    --config config/inference_config.yaml

import torch
import torchaudio
from confuciustts.cli.inference import ConfuciusTTS

model = ConfuciusTTS(
    config_path="config/inference_config.yaml",
    device="cuda" if torch.cuda.is_available() else "cpu",
)

audio = model.generate(
    text="Hello, welcome to Confucius4-TTS.",
    lang="en",
    prompt_wav="path/to/reference.wav",
    verbose=True,
)

torchaudio.save("output.wav", audio.cpu(), model.sample_rate)

Tool | What it does
--prompt_wav | The reference recording whose voice is copied
--text | The text to speak
--lang | The language code, such as en
--out | The output audio file
--config | The inference settings file

Example: You point the script at a reference recording you have the right to use and ask it to say "Hello, welcome to Confucius4-TTS." with the language set to en. It saves output.wav with that sentence spoken in the reference voice.

For newcomers and experts: If you are new to this, it reads text aloud and can borrow the sound of a short sample voice. A first small step is to run the example script with one reference clip and one short sentence.

If you know the field, it uses a speech encoder plus LLM design with two modules. Text2Semantic generates semantic tokens from text and speaker conditioning, and Semantic2Acoustic is a flow-matching model that turns those tokens into mel spectrograms. Fine-tuning needs two external models, Wav2Vec2-BERT and Amphion MaskGCT.

Text and a reference voice go in
Text2Semantic makes semantic tokens
Semantic2Acoustic builds mel spectrograms
Speech audio comes out

Good to know: The card says it supports 14 languages, among them Chinese, English, Japanese, Korean and German, and that more are coming.

• License: Apache-2.0
• Cost to run: no API key or paid service is named, but the card asks for CUDA 12.6 and a GPU setup
• Data it reaches: the download comes from Hugging Face and GitHub, and the model works on the text and reference audio you give it
• Not for: cloning a voice you have no right to use

On Hugging Face, 13 November 2026

• Task: text to speech
• Formats: Safetensors
• Downloads in the last 30 days: 2,190
• Likes: 46
• License: apache-2.0
• Languages: zh, en, ja, ko, de, fr, id, vi
• Quantized versions: 1
• Fine-tunes: 4
• Tags: safetensors, confucius4, text-to-speech, speech-synthesis, zero-shot, voice-cloning, cross-lingual, multilingual

Links

• Model: https://huggingface.co/netease-youdao/Confucius4-TTS
• Files: https://huggingface.co/netease-youdao/Confucius4-TTS/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download netease-youdao/Confucius4-TTS

Then follow the usage in its model card. Nothing in this kit runs by itself.