Pipecat Tone

Pipecat Tone

Gives Pipecat voice agents two small CPU models, one that reads a caller's emotion and sarcasm and one that tags the agent's reply for expressive speech. It suits developers of voice agents.

Made by pipecat-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Pipecat Tone is a pair of small CPU models for Pipecat voice agents. The tone reader judges how a user turn was meant, giving an emotion and a sarcasm check with calibrated confidences. The tone tagger adds expression tags to the agent's plain reply, one sentence at a time, for the speech engine to perform.

Who it suits: It suits developers who build voice agents with Pipecat and want them to hear tone and answer with more expression. Both models run on a CPU in milliseconds.

Industry | Who uses it | How they use it | Example
Voice AI | Pipecat developers | Add tone reading and expressive replies to an agent | An agent that apologizes in a softer voice
Customer support | Support automation teams | Let an agent notice anger or sarcasm in a caller | Spotting a sarcastic remark about a long wait
Software | Speech product builders | Shape how a text-to-speech voice delivers a reply | Marking slow spans for an order number

People who want to judge or profile the people a voice agent talks to should not use it, since the card rules that out.

Where to use it:

• Pipecat 1.15.0 or later
• Voice agent pipelines with speech-to-text, an LLM and text-to-speech
• A CPU, on Windows, macOS or Linux
• English-language conversations only

Use case | Who | What they get back
Reading tone | Voice agent developers | An emotion and a sarcasm value, each with a score for how sure it is
Expressive replies | Speech developers | Tags for delivery, slow spans, stressed words and breaths
Context for the LLM | Agent builders | A note on how the user spoke, placed before their message

How to set it up:

• Pipecat 1.15.0 or later
• The pipecat-ai package with its tone extra
• A text-to-speech voice that performs the expression tags

1. Install the Hugging Face hub package, if you want the files on disk.
2. Install the pipecat-ai package with the tone extra.
3. Turn on the expressive conversation setting and add the two processors.

pip install -U huggingface_hub

hf download pipecat-ai/tone-models

How to use it: Turn on expressive conversation in the pipeline settings, then place the reader after speech-to-text and the tagger after the LLM. The models download from this repository the first time a pipeline starts. Pass size 32m to the tagger to use the larger one. Breaths need the BREATHE guidance in the expression policy.

from pipecat.processors.tone.reader import PipecatToneReader
from pipecat.processors.tone.tagger import PipecatToneTagger

pipeline = Pipeline([
    transport.input(), stt, PipecatToneReader(), user_aggregator,
    llm, PipecatToneTagger(), tts, transport.output(), assistant_aggregator,
])

Tool | What it does
PipecatToneReader | Reads the emotion and sarcasm of each user turn
PipecatToneTagger | Writes expression tags for the agent's reply
size 32m | Picks the larger tagger, which scores a little higher and runs slower

Example: A caller says something sarcastic about a long wait. The reader marks the turn as angry with a high sarcasm value, and the LLM sees that note before the words. The LLM answers in plain text, and the tagger adds tags such as an apologetic delivery and a light breath for the voice.

For newcomers and experts: Voice agents usually pass only words between their parts, so tone gets lost. These models carry tone along in the same text stream. A first small step is to add the two processors to an existing Pipecat pipeline and listen to the difference.

The card calls this the Real-time Context Standard. The reader is a MiniLM-based classifier, and the tagger is a small Ettin model with six heads. The reader's p95 time per turn is 5.0 ms on four CPU threads on an Apple M1 Max.

User speech becomes text
Reader adds emotion and sarcasm notes
LLM writes a plain reply
Tagger adds tags, voice performs them

Good to know:

• License: BSD 2-Clause.
• Cost to run: none named. Both models run on a CPU.
• Data it reaches: the models download from Hugging Face the first time a pipeline starts, and then process the conversation you run through it.
• Not for: judging or profiling the people the agent talks to. The card says it only shapes how the voice sounds.
• English only, since it learned from support, assistant and meeting conversations.

Title: Tagger combined score by model size
17M tagger: 0.753 score
32M tagger: 0.767 score

On Hugging Face, 13 November 2026

• Task: text classification
• Library: onnx
• Downloads in the last 30 days: 0
• Likes: 10
• License: bsd-2-clause
• Languages: en
• Based on: sentence-transformers/all-MiniLM-L6-v2
• Tags: onnx, pipecat, voice-agents, emotion-recognition, expressive-tts, text-classification

Links

• Model: https://huggingface.co/pipecat-ai/tone-models
• Files: https://huggingface.co/pipecat-ai/tone-models/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download pipecat-ai/tone-models

Then follow the usage in its model card. Nothing in this kit runs by itself.