Nemotron 3 Diarization

Nemotron 3 Diarization

Works out who spoke when in a recording, with up to eight speakers, in streaming or offline mode, and can add speaker labels to a transcript. For people who process meeting, call and media audio.

Made by nvidia on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Nemotron 3 Diarization is an open-weight NVIDIA model that labels the speakers in audio, answering who spoke when. It handles up to eight speakers and runs either on a live stream or on a finished recording. One checkpoint covers both, with an input buffer that can be as short as 80 ms or as long as 30.4 s.

Who it suits: It suits developers and teams who work with multi-speaker audio and want speaker labels without building their own pipeline.

Industry | Who uses it | How they use it | Example
Meetings and calls | Analytics teams | Split a recording by speaker | Seeing which person spoke in each stretch of a call
Transcription | Speech app developers | Add speaker tags to a transcript | Producing a transcript with a speaker label on every word
Media | Editors and producers | Mark speaker turns in interviews and panels | Finding where each guest talks
Speech research | Researchers | Study streaming diarization with a configurable delay | Testing different buffer sizes on the same clip

You can skip it if your audio has more than eight speakers at once, or if you need transcription alone.

Where to use it:

• NeMo-Speech.cpp, a lightweight native C++ runtime for local use
• NVIDIA NeMo Speech in Python, for inference, training and fine-tuning
• A Hugging Face Spaces demo that pairs it with streaming transcription
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Diarize a recording | Developers | Speaker segments showing who spoke when
Speaker-tagged transcript | App builders | Word-level speaker tags added to a transcription
Live diarization | Product teams | Speaker labels from a stream at low latency

How to set it up: You choose between the C++ runtime and the Python toolkit. The Python path calls for a Python version of at least 3.12, plus Cython and an up-to-date PyTorch build.

• Python 3.12 or later for the NeMo route
• Cython, PyTorch, libsndfile and ffmpeg
• Or the NeMo-Speech.cpp runtime, installed from its own guide
• A Hugging Face token if you load the model straight from the Hub in Python

1. Pick a runtime, either NeMo-Speech.cpp or the NeMo Python toolkit.
2. For the Python route, install the system packages and the toolkit with the commands below.
3. Download the model, or let the script load it from the Hub.

apt-get update && apt-get install -y libsndfile1 ffmpeg
uv pip install Cython packaging
uv pip install 'nemo-toolkit[asr]'

pip install -U huggingface_hub
hf download nvidia/Nemotron-3-Diarization

How to use it: With NeMo-Speech.cpp you run a command on a WAV file. The second command also asks for a transcription with speaker tags and JSON output.

nemo-speech diarize meeting.wav

nemo-speech transcribe meeting.wav --diarize --json

In Python, the card's quick start loads the model, sets the streaming chunk values, diarizes a WAV file and prints each segment.

from nemo.collections.asr.models import SortformerEncLabelModel
diar_model = SortformerEncLabelModel.from_pretrained("nvidia/Nemotron-3-Diarization")
diar_model.eval()

diar_model.sortformer_modules.chunk_len = 340
diar_model.sortformer_modules.chunk_right_context = 40
diar_model.sortformer_modules.fifo_len = 40
diar_model.sortformer_modules.spkcache_update_period = 300
diar_model._check_streaming_parameters()

predicted_segments = diar_model.diarize(audio=["/path/to/your/audio.wav"], batch_size=1)

for segment in predicted_segments[0]:
    print(segment)

Tool | What it does
nemo-speech diarize | Labels who spoke when in a WAV file
nemo-speech transcribe --diarize --json | Transcribes the audio with speaker tags, in JSON form
SortformerEncLabelModel | Loads the model in the NeMo Python toolkit
diarize | Returns speaker segments for a list of audio files

Example: You run nemo-speech diarize on meeting.wav. The tool prints the speaker segments, showing which speaker was talking over each stretch of the recording.

For newcomers and experts: If you are new to this, diarization means labeling a recording by speaker, like Speaker 1 and Speaker 2. A first small try is the single diarize command on a short WAV file.

For people who know the field, the model follows Sortformer and fixes the speaker permutation problem by ordering outputs by arrival. For streaming it uses an arrival-order speaker cache and a FIFO queue. With chunked inference the audio length has no cap.

Audio comes in as a stream or a file
The model reads it in chunks
A speaker cache keeps identities steady over time
You get speaker segments, up to eight speakers

Good to know: The lowest input buffer latency is 80 ms, though the card names 0.32 s as the lowest recommended setting. Output frame resolution can be set in multiples of 10 ms.

• License: openmdw-1.1, with the card allowing both commercial and non-commercial use
• Cost to run: none named
• Data it reaches: the audio files or streams you pass in, plus the downloaded model files
• Not for: recordings with more than eight speakers

On Hugging Face, 13 November 2026

• Task: voice activity detection
• Library: nemo
• Parameters: 99.2M
• Weights: about 397 MB in F32; about 50 MB at 4-bit (estimate)
• Formats: Safetensors, GGUF
• Downloads in the last 30 days: 204,726
• Likes: 836
• License: openmdw-1.1
• Quantized versions: 39
• Fine-tunes: 16
• Tags: nemo, safetensors, gguf, nemotron3_diarization, audio-frame-classification, speaker-diarization, streaming-sortformer, speaker-tagging, transformers, voice-activity-detection

Links

• Model: https://huggingface.co/nvidia/Nemotron-3-Diarization
• Files: https://huggingface.co/nvidia/Nemotron-3-Diarization/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download nvidia/Nemotron-3-Diarization

Then follow the usage in its model card. Nothing in this kit runs by itself.