Full Duplex Leaderboard

Full Duplex Leaderboard

Compares voice agents on when to speak, which tool to call and how well speech follows a style instruction, using three public benchmarks. For teams that build or choose voice agents.

Made by aufklarer on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: This leaderboard scores voice agents on three questions, each with its own tab. When should the agent start speaking, which tool should it call for a spoken command, and does its speech follow a style instruction. The results come from three public benchmark datasets, with the scoring methods stored in the repository.

Who it suits: It suits researchers and engineers who build or pick speech agents and want measured numbers instead of claims. It also suits readers who want to see how each benchmark is scored.

Industry | Who uses it | How they use it | Example
Voice AI | Agent builders | Compare turn-taking behavior across systems | Check the turn-taking tab for each system's recall and false positives
Software | Voice-app developers | Pick a model for command handling | Read the commands table for tool-routing results
Speech research | TTS researchers | Compare style-following in generated speech | Read the style table next to the human recordings
Evaluation | Benchmark authors | Study how the scores are built | Open the protocol files in the repository

Anyone who wants a general chatbot ranking can skip it, since these tests cover only voice behavior.

Where to use it: The leaderboard runs in two places.

• A web browser, on the page of the Space at Hugging Face
• Windows, macOS or Linux, as a local copy run with Python

Use case | Who | What they get back
Turn-taking check | Agent builders | A ranking by recall, false positives and detection latency
Tool-calling check | Developers | Counts of calls made despite a missing tool
Voice-style check | Researchers | The share of style checks each speech system passes

How to set it up: The online page needs nothing installed. A local copy needs the following.

• Python and pip
• Windows, macOS or Linux
• The packages listed in requirements.txt

1. Install Python.
2. Install the Hugging Face hub package and download a copy of the Space.
3. Move into the new folder and install Gradio 5.50.0 and the packages in requirements.txt.
4. Start the app with python app.py.

pip install -U huggingface_hub
hf download aufklarer/full-duplex-leaderboard --repo-type space --local-dir full-duplex-leaderboard
cd full-duplex-leaderboard
pip install gradio==5.50.0
pip install -r requirements.txt
python app.py

How to use it: Open the page and move between its tabs. Each tab shows one benchmark's table. On a local copy, this command starts the page.

python app.py

Tool | What it does
Turn-taking tab | Ranks systems on when to speak, using the TurnBench dev split
Commands tab | Scores tool routing, using the SLURP test split
InstructTTS tab | Scores whether speech follows its style instruction
results folder | Holds the result rows for each track
protocol folder | Holds the full scoring protocols

Example: You open the turn-taking tab to see which systems decide best when to speak. The table ranks them by a single score and also shows recall, false positives and detection latency separately.

For newcomers and experts: If you are new to this, start with the turn-taking tab. It asks a simple thing, whether an agent speaks at the right moment, and the table explains the columns.

If you know the field, turn-taking is ranked by Youden's J, which is recall minus false-positive rate, shown multiplied by 100. The commands score is the authors' own routing metric built from SLURP, not SLURP's false-activation score. The voice-style tab reports pass counts over 150 bilingual style checks for some systems, rated with Gemini 3.1 Pro, and English-only pilots show a smaller sample.

Three public benchmarks
Systems are run and measured
Rows are saved as JSON files
Tabs show ranked tables

Good to know: The authors list only configurations they ran themselves. Rows live in results files, display names in a roster folder, and the combined catalogue in a data file.

• License: MIT for the Space
• Cost to run: none named
• Data it reaches: the benchmark data and result files in the repository, plus the InstructTTSEval dataset the Space names
• Data licences: TurnBench is for non-commercial use with no voice cloning, and SLURP text is CC BY 4.0 while its audio is CC BY-NC 4.0
• Not for: treating the commands score as SLURP's own false-activation or background-speech measure

On Hugging Face, 13 November 2026

• Likes: 2
• Built with: gradio
• License: mit

Links

• Space: https://huggingface.co/spaces/aufklarer/full-duplex-leaderboard
• Files: https://huggingface.co/spaces/aufklarer/full-duplex-leaderboard/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.