caramel-1 by naklecha

caramel-1 by naklecha

Listens and speaks in a live back-and-forth as a 413M-parameter speech-to-speech model trained only on recordings, never on text. For voice AI researchers and tinkerers.

Made by naklecha on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: caramel-1 is a speech-to-speech conversation model inspired by Dessa. It learned only from recordings of people talking and was never shown text. Its code, training recipe and weights are all published openly. The author says it sounds human, but most of what it says does not make sense yet.

Who it suits: It suits researchers and tinkerers who want to study how a small model trained on speech alone handles a live conversation.

Title: Who might use it
Industry | Who uses it | How they use it | Example
Voice AI | Researchers | Study how a speech-only model takes turns | Two copies deciding when to speak or stay quiet
Speech data | Data teams | Learn from the data sources and the scoring method | A closeness score built from 13 timing and recording statistics
Education | Students | Read an open recipe for a small speech model | Following how audio becomes tokens at 12.5 frames per second

People who need a chatbot that makes sense can skip it, since the author says most replies do not yet.

Where to use it:

• PyTorch: the library the model is published for
• A GPU: the card says audio comes out about 11 times faster than real time
• A MacBook Pro (M3 Pro): the card reports both copies together at roughly four times real-time speed
• Windows, macOS or Linux: download the files with Python

Title: Jobs it does
Use case | Who | What they get back
Live conversation demos | Researchers | Two copies talking to each other as they listen and speak
Turn-taking studies | Speech teams | A model that decides every 80 ms whether to speak
Data and scoring ideas | Data teams | A method for rating how close a conversation is to a target

How to set it up: You need these first:

• Python on Windows, macOS or Linux
• PyTorch, the library the model uses
• A GPU or a recent Mac for real-time speed
• About 826 MB of storage for the BF16 weights

1. Install the Hugging Face hub package.
2. Download the model files.
3. Run the model with the code and steps the card points to.

pip install -U huggingface_hub
hf download naklecha/caramel-1

How to use it: Download the weights, then follow the card and its GitHub project to run the model. This command fetches the files.

hf download naklecha/caramel-1

The card describes running two copies side by side. Each copy listens to the other every 80 ms and decides whether to talk or stay silent. Its sample transcripts come from a speech recogniser and only help you follow along.

Example: You run two copies and let them listen to each other. You hear a spoken back-and-forth that sounds human, though most of what is said does not make sense yet.

For newcomers and experts: If you are new to this, think of it as a voice that learned to talk from recordings alone. Download the 826 MB of weights and read the card's sample description before you run anything.

If you work in the field, a neural audio codec converts each speaker's audio to 12.5 frames per second, and every frame holds 8 tokens. That is why the model never handles raw audio. Training covered roughly 33B tokens from 11,555 hours of speech. According to the card, adding hours and passes lowered the loss on conversations the model had not seen. The author scores conversations against 13 statistics taken from Dessa examples.

Each speaker's audio becomes codec tokens
The model predicts what comes next
Every 80 ms it speaks or stays quiet
Generated tokens become speech

Good to know: The card lists how much speech the model learned from.

Title: Hours of training speech
Two-person conversations: 5743 hours
One-person speech: 5812 hours

• License: apache-2.0
• Cost to run: none named
• Data it reaches: the audio you feed it, while the training data was public speech from the internet
• Not for: dependable conversation, since the author says most of what it says does not make sense yet

On Hugging Face, 13 November 2026

• Task: audio to audio
• Library: pytorch
• Parameters: 413M
• Weights: about 826 MB in BF16; about 207 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 89
• Likes: 14
• License: apache-2.0
• Fine-tunes: 1
• Tags: pytorch, safetensors, audio, speech, speech-to-speech, full-duplex, spoken-dialogue, audio-to-audio

Links

• Model: https://huggingface.co/naklecha/caramel-1
• Files: https://huggingface.co/naklecha/caramel-1/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download naklecha/caramel-1

Then follow the usage in its model card. Nothing in this kit runs by itself.