# Voice cloning with AI: how it works, what consent means and which models to use

> Voice cloning makes a synthetic voice that sounds like a real speaker from a short recording. Open models such as XTTS-v2 and Chatterbox work from a short reference clip (XTTS-v2's card says about 6 seconds), while OpenAI asks for a recorded consent phrase and limits recorded voices to eligible customers. Clone only your own voice or one whose owner agreed, and tell listeners the audio is synthetic.

Mindket editors · Published 2026-11-06 · Updated 2026-11-06 · https://mindket.com/learn/ai-voice-music/voice-cloning

## In short

- Describe a voice in words when no particular person has to be copied.
- Record a clean sample, and only of a voice you own or whose owner agreed.
- Read the license on a model's weights before you sell anything it makes.
- Keep any watermark in the file and label the clip as synthetic.
- Treat a clean provenance check as no answer rather than a yes.

## Before you start

**Voice cloning** means building a voice from a recording of one particular person, so new text comes out sounding like them. The open-model examples below have no training step: you give a short **reference clip** and new text, and the model speaks the text in that voice. Chatterbox's README calls this **zero-shot**. The [text to speech lesson](https://mindket.com/learn/ai-voice-music/text-to-speech) covers choosing a built-in voice, and this one starts where that stops.

A **described voice** comes from a text description, and nobody is copied. A **cloned voice** comes from a recording of a real person, so it carries that person's likeness, and every rule in this lesson follows from that. The steps below need an OpenAI API key for the hosted route and Python for the open models.

## No recording needed: describe a voice

OpenAI's prompt-to-voice creates and saves a voice from a description, with no reference recording and no consent recording. It is generally available to all API customers. The saved voice works with GPT-Live, and the guide says it is not supported by the Text to Speech, Realtime or Chat Completions APIs. Creating one is free, and using it in GPT-Live is billed at regular session rates.

You describe qualities, not a person. The guide lists nine, and they are words in your prompt rather than separate API parameters. It also says not to ask for imitation of a named person, a public figure or a fictional character, and that children's voices are not allowed.

````bash
curl --fail-with-body https://api.openai.com/v1/audio/voices \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "prompt",
    "name": "Support assistant",
    "prompt": "A calm, reassuring adult voice for a customer support assistant. Speak with an American English accent, a mid-range pitch, a smooth, warm tone, and a steady, conversational pace.",
    "model": "auto"
  }' \
  -o voice.json
````

1. Write the description from the qualities in the table below, in plain language.
2. Send the request above. Each successful request saves a new voice and returns its metadata, including an `id` such as `voice_123`, with no audio or preview.
3. Keep the `id` and the original description. The guide says to reuse a saved voice when you want the same voice in another session.
4. Listen in a GPT-Live session by setting `session.audio.output.voice.id` to that `id`. For voice performance, the guide says to include the description in `session.instructions` as well.
5. Change one quality at a time and compare using the same short passage.

| Quality | What to describe | Example wording |
| --- | --- | --- |
| Accent | Regional pronunciation and accent strength | American English, with a strong accent |
| Age | The perceived age of the voice | An adult in their 30s |
| Presentation | How the voice presents | Feminine |
| Pitch | How low or high the voice sounds | Mid-range pitch |
| Timbre | The tone of the voice | Smooth and rich |
| Texture | Raspiness, hoarseness or clarity | Smooth |
| Loudness | Vocal effort | Moderate vocal effort |
| Speech rate | How quickly the voice speaks | Medium pace |
| Distinctiveness | How pronounced the voice's individual character should be | Moderately distinctive |

## Clone a voice from a recording: OpenAI's consent flow

A voice from a recording is limited to eligible customers. The guide says to contact OpenAI's sales team, and once your organization is enabled a Voices tab appears under Audio. It takes two separate recordings from the same person: a consent recording, where the speaker reads one set phrase, and a sample recording, which the model tries to copy.

The English phrase is: "I am the owner of this voice and I consent to OpenAI using this voice to create a synthetic voice model." The guide lists the phrase for 16 languages, including Chinese, and warns that any divergence from the script makes the upload fail. One consent can be reused when the same speaker makes more than one voice.

**Limits the guide states**

- **60 s** the longest sample OpenAI accepts for a recorded voice (Source: [OpenAI API docs: Custom voices](https://developers.openai.com/api/docs/guides/custom-voices), read 2026-11-06)
- **20** recorded voices an organization can create (Source: [OpenAI API docs: Custom voices](https://developers.openai.com/api/docs/guides/custom-voices), read 2026-11-06)
- **100** described voices an organization can save, a limit that also covers voices your app creates for its customers (Source: [OpenAI API docs: Custom voices](https://developers.openai.com/api/docs/guides/custom-voices), read 2026-11-06)

````bash
curl https://api.openai.com/v1/audio/voice_consents \
  -X POST \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F "name=test_consent" \
  -F "language=en" \
  -F "recording=@$HOME/tmp/voice_consent/consent_recording.wav;type=audio/x-wav"

curl https://api.openai.com/v1/audio/voices \
  -X POST \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F "name=test_voice" \
  -F "audio_sample=@$HOME/tmp/voice_consent/audio_sample_recording.wav;type=audio/x-wav" \
  -F "consent=cons_123abc"
````

1. Get access. Without it, the voice endpoints answer with an error such as a 404.
2. Record the consent phrase exactly as written, in a quiet room.
3. Upload it to `/v1/audio/voice_consents` with a name and its language. The response holds a consent ID.
4. Record the sample, 60 seconds or less, in one of mpeg, wav, ogg, aac, flac, webm or mp4.
5. Create the voice at `/v1/audio/voices` with the sample and the consent ID.
6. Use the voice's ID in a Realtime session with `gpt-realtime-2.1-mini`, or in GPT-Live. Choose it at the start, because a different voice needs a new session.

## What the sample has to be

The guide says the sample sets the quality. Its advice is a quiet space with little echo, a professional XLR microphone, and a spot about 7 to 8 inches from the mic with a pop filter in between, kept the same throughout. The model copies exactly what it hears, tone, cadence, energy, pauses and habits, so record the voice you want the clone to have. A sample that wanders in energy or accent wanders in the result.

The open models give less advice. Chatterbox's README warns that a reference clip in a different language from the target can pass its accent along, and suggests setting `cfg_weight` to `0` to reduce that. XTTS-v2's card names 6 seconds as the clip length. The advice above comes from OpenAI's guide, but a quiet room and a steady delivery cost nothing, so keep them for any clip you record.

## Clone on your own computer with an open model

Open models need no approval and send nothing to a vendor. None of them has a consent step, so the duty sits with you. F5-TTS is one with a plain command line. It takes a reference recording and, if you have it, the words spoken in that recording, then speaks new text in that voice. Leave the reference text empty and an ASR model transcribes the recording, at the cost of extra GPU memory.

Chatterbox does the same through Python: `model.generate(text, audio_prompt_path="your_10s_ref_clip.wav")`, where the README's example file is named for a 10-second clip, though the README states no length. Its README says the project was retired in August 2026 and is no longer maintained, so the code stays public as an archive. The commands look alike. The licenses on the weights do not.

````bash
pip install f5-tts

# Leave --ref_text "" will have ASR model transcribe (extra GPU memory usage)
f5-tts_infer-cli --model F5TTS_v1_Base \
--ref_audio "provide_prompt_wav_path_here.wav" \
--ref_text "The content, subtitle or transcription of reference audio." \
--gen_text "Some text you want TTS model generate for you."
````

**Three open models that clone from a clip**

| | XTTS-v2 | Chatterbox | F5-TTS |
| --- | --- | --- | --- |
| What it needs from you | A clip of about 6 seconds | A short clip, with no length stated in the README (its example file is named 10s) | A reference recording and, optionally, its text |
| Languages | 17 | 23+ with Multilingual V3 | Emilia Chinese and English model, more through custom inference |
| License of the weights | Coqui Public Model License | MIT, per its Hugging Face card | CC-BY-NC, non-commercial |
| Runs on your own computer | ✓ Yes | ✓ Yes | ✓ Yes |
| Marks what it makes | Not stated on its card | Yes, a PerTh watermark in every file | Not stated in its README |

Sources: [Hugging Face: coqui/XTTS-v2 model card](https://huggingface.co/coqui/XTTS-v2) (read 2026-11-06), [GitHub: resemble-ai/chatterbox README](https://github.com/resemble-ai/chatterbox) (read 2026-11-06), [Hugging Face: ResembleAI/chatterbox model card](https://huggingface.co/ResembleAI/chatterbox) (read 2026-11-06), [GitHub: SWivid/F5-TTS README](https://github.com/SWivid/F5-TTS) (read 2026-11-06)

## The rules before you clone anything

Laws about a person's voice differ by country and by state, and this lesson does not cover them. Ask a lawyer before you publish or sell cloned audio. The vendors' own rules are easier to read. OpenAI's pages repeat three of them, and they make a fair baseline for any clone: the speaker consents, nobody is impersonated, and listeners are told.

Under prompt-to-voice, OpenAI's responsible-use list says: use voices responsibly, do not impersonate others, use adult voices, and tell listeners that the speech is AI-generated while preserving any provenance metadata or watermarks. It also says voices are not guaranteed to be unique, since others may generate similar ones. The guide prints that list under described voices. For recorded voices it asks for the consent recording and points to the Text-to-Speech Supplemental Agreement for more terms. Holding a recorded voice to the whole list is this lesson's advice, not a line the guide prints there.

> Keep the speaker's written agreement next to the audio files, and put a line in the description that says the voice is synthetic.

| Who | What it asks or does |
| --- | --- |
| OpenAI, recorded voices | A consent recording of the speaker, read from one set phrase, and a sample from the same person. The guide points to its Text-to-Speech Supplemental Agreement for more terms |
| OpenAI, described voices | Its responsible-use list asks for adult voices, no impersonation, a notice that speech is AI-generated and provenance signals kept. Its prompt tips rule out naming a real person, a public figure or a character |
| Chatterbox | Puts a PerTh watermark in every file, and its README asks you not to use the model to do bad things |
| Kokoro | Its card says training used no synthetic audio from open TTS models or "custom voice clones", which describes its training data, not what the model can do |
| XTTS-v2 and F5-TTS | Their pages set a license on the weights and say nothing about consent, so that duty stays with you |

## Checking a clip, and what a clean result means

OpenAI's Content Provenance API checks an image or audio file for its own provenance signals. Send the file to `POST /v1/content_provenance_checks`, or use the web tool at openai.com/verify. For audio, the signal is SynthID, a watermark embedded in the audio itself that may survive some edits. C2PA Content Credentials are signed metadata on images, and editing, converting or sharing a file can strip them.

A `not_detected` result means the tool found no supported signal. The clip can still come from OpenAI if its metadata was stripped, its watermark degraded, it came from a legacy model or it predates the signals. The tool does not detect other companies' models either, and the page says it is not a general-purpose AI detector. Chatterbox's own watermark has a separate check: its README shows a short script, `perth.PerthImplicitWatermarker()` followed by `get_watermark`, that returns 1.0 for watermarked audio and 0.0 for none.

| Signal | Applies to | What it checks |
| --- | --- | --- |
| C2PA Content Credentials | Images | Signed metadata with issuer and AI-use details |
| SynthID | Images and audio | A watermark embedded directly in supported media |

## Common mistakes

- Cloning a voice without the speaker's agreement because the tool did not ask for one.
- Reading the consent phrase from memory instead of from the script.
- Recording a sample in a noisy or echoing room, or in a different mood from the one you want, since the model copies both.
- Using F5-TTS's CC-BY-NC weights for paid work.
- Setting a reference clip in one language and the target text in another, then wondering about the accent.
- Treating `not_detected` as proof that a clip is human.
- Expecting a described voice to work in the Text to Speech API when the guide limits it to GPT-Live.

## Try it yourself

- Write a description for a voice, using three of the nine qualities, and change one of them to see what moves.
- Read the F5-TTS command and the Chatterbox line above, and write down which of the two weights you could use in a paid project.
- Record a short clip of your own voice in a quiet room and clone it with an open model, then label the result as synthetic.
- Run a short audio file through the provenance web tool at openai.com/verify and note what a `not_detected` result does and does not tell you.

## Next lesson: AI music

Voices say words. [AI music](https://mindket.com/learn/ai-voice-music/ai-music) turns a style and some lyrics into a whole song, with its own licensing questions about the models, the hosted tools and what you may release.

## Check yourself

1. What must the speaker do before OpenAI will make a voice from a recording?

   A. Sign a paper form
   B. Read a set consent phrase in a separate recording
   C. Record 5 minutes of speech
   D. Upload a photo ID

   Answer: B. The guide asks for a consent recording in which the speaker reads one of its phrases, and says any divergence from the script makes it fail.

2. How long a clip does XTTS-v2's card say is enough to clone a voice?

   A. About 60 seconds
   B. About 30 minutes
   C. About 6 seconds
   D. A full hour

   Answer: C. The card says it clones voices into different languages from a quick 6-second audio clip, with no need for an excessive amount of training data that spans countless hours.

3. Which statement about F5-TTS is right?

   A. Its pre-trained models are CC-BY-NC, so non-commercial
   B. Its weights are MIT-licensed
   C. It adds a PerTh watermark to every file
   D. It needs OpenAI approval

   Answer: A. Its README puts the code under MIT but licenses the pre-trained models CC-BY-NC, because of the Emilia training data.

4. OpenAI's provenance check returns not_detected for an audio clip. What does that show?

   A. The clip is human
   B. The clip is safe to publish
   C. The clip came from another company
   D. Only that no supported signal was found

   Answer: D. The page says stripped metadata, a degraded watermark, an older model or another company's model can all return not_detected.

## Frequently asked questions

### Can AI clone my voice from a few seconds of audio?

Yes, with some open models. XTTS-v2's card says a quick 6-second audio clip is enough, and the Chatterbox README names its example file your_10s_ref_clip.wav without stating a length. OpenAI accepts up to 60 seconds for a recorded voice, and its guide says the quality of the sample sets the quality of the voice.

### Is it legal to clone someone's voice?

That depends on where you live and how the clone is used, and this lesson gives no legal advice. What the vendors ask is clearer. For a recorded voice OpenAI requires a consent recording from the speaker, and its responsible-use list for described voices says adult voices only and no impersonation, so get the speaker's agreement first.

### Can I clone a celebrity's voice?

Not through OpenAI. A described voice cannot ask for a named person, public figure or fictional character, and a recorded voice needs the owner's own spoken consent. The open models' pages show no such gate, so the decision falls on you.

### Which open-source voice cloning model is free for commercial use?

Chatterbox's README puts its code under MIT and its Hugging Face card lists MIT as the license, though the project is retired. XTTS-v2 uses the Coqui Public Model License, whose terms you should read before selling output. F5-TTS code is MIT, but its pre-trained models are CC-BY-NC, which is non-commercial.

### How can I tell whether audio was made by AI?

For OpenAI audio, upload it to openai.com/verify or send it to the Content Provenance API, which looks for SynthID. A not_detected answer proves little, because stripped metadata, a degraded watermark and other companies' models all pass it.

## Sources

- [OpenAI API docs: Custom voices](https://developers.openai.com/api/docs/guides/custom-voices) (read 2026-11-06)
- [OpenAI API docs: Content provenance](https://developers.openai.com/api/docs/guides/content-provenance) (read 2026-11-06)
- [Hugging Face: coqui/XTTS-v2 model card](https://huggingface.co/coqui/XTTS-v2) (read 2026-11-06)
- [GitHub: resemble-ai/chatterbox README](https://github.com/resemble-ai/chatterbox) (read 2026-11-06)
- [Hugging Face: ResembleAI/chatterbox model card](https://huggingface.co/ResembleAI/chatterbox) (read 2026-11-06)
- [GitHub: SWivid/F5-TTS README](https://github.com/SWivid/F5-TTS) (read 2026-11-06)
- [Hugging Face: SWivid/F5-TTS model card](https://huggingface.co/SWivid/F5-TTS) (read 2026-11-06)
- [Hugging Face: hexgrad/Kokoro-82M model card](https://huggingface.co/hexgrad/Kokoro-82M) (read 2026-11-06)

[Compare voice tools](https://mindket.com/learn/ai-tools/best-ai-voice-generator)