SPEAR XLarge v2

SPEAR XLarge v2

Turns 16 kHz speech and general audio into 1280-number frame-level features, as a pretrained Zipformer encoder with about 600M parameters. For researchers who fine-tune it for speech recognition or audio tagging.

Made by marcoyang on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: SPEAR XLarge v2 is a pretrained audio encoder that learns from speech and general sounds in one model. It turns a 16 kHz waveform into 1280-number features at about 50 per second. It uses a Zipformer backbone with 13 stacks, and version 2 adds token mixing for overlapped speech, noise and mixed sound scenes.

Who it suits: Researchers and engineers who need a starting point for speech recognition, audio tagging or other audio tasks, and who can fine-tune on labeled data.

Industry | Who uses it | How they use it | Example
Speech technology | ASR engineers | Fine-tune the encoder for speech recognition | An encoder tuned on LibriSpeech
Audio analysis | Sound researchers | Fine-tune it to tag audio events | A tagger tuned on AudioSet
Academia | Students | Extract features for experiments on speech and audio | Hidden states from each layer for a probe
Media | Audio engineers | Study mixed scenes with speech, noise and music | Features from a noisy recording

Skip it if you need a ready transcriber, because it was pretrained on unlabeled audio only and needs fine-tuning for supervised tasks.

Where to use it:

• Python with PyTorch and the transformers library
• A CPU or a CUDA GPU, which the example chooses automatically
• Research pipelines on Windows, macOS or Linux

Use case | Who | What they get back
Feature extraction | Researchers | Frame-level features with shape batch, frames, 1280
Intermediate layers | Probing studies | A list of hidden states, 13 layers in all
Fine-tuning base | ASR and tagging teams | A pretrained encoder to train on labeled data

How to set it up: You need Python with pip, PyTorch, the transformers library and about 2.4 GB of disk space for the F32 weights. Your audio must be resampled to 16 kHz before it goes in.

• Python and pip
• PyTorch and transformers
• About 2.4 GB of free disk space

1. Install the Hugging Face hub tool.
2. Download the repository's files, or let transformers fetch them on first load.
3. Resample any audio to 16 kHz.

pip install -U huggingface_hub
hf download marcoyang/spear-xlarge-speech-audio-v2

How to use it: Load the model with transformers, passing the option that allows the repository's own code to run. Then give it a mono or batched 16 kHz waveform and its length. The result holds the top-layer features, their lengths and all hidden states.

from transformers import AutoModel
import torch

model = AutoModel.from_pretrained(
    "marcoyang/spear-xlarge-speech-audio-v2",
    trust_remote_code=True,
)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device).eval()

# Dummy 10-second waveform at 16 kHz.
audio = torch.randn(1, 160000, device=device)
audio_len = torch.tensor([audio.shape[1]], device=device)

with torch.no_grad():
    outputs = model(audio, audio_len)

encoder_out = outputs["encoder_out"]       # (batch, frames, 1280)
encoder_out_lens = outputs["encoder_out_lens"]
hidden_states = outputs["hidden_states"]   # list of intermediate states

print(encoder_out.shape)
print(encoder_out_lens)
print(len(hidden_states))                  # 13 layers
print(hidden_states[-1].shape)

Tool | What it does
AutoModel.from_pretrained | Loads the encoder from the repository
encoder_out | The top-layer features for each frame
hidden_states | A list of the intermediate layer outputs

Example: You pass a ten-second clip at 16 kHz and its length to the model. It returns frame features with 1280 numbers each, their lengths, and a list of 13 hidden states you can inspect.

For newcomers and experts: If you are new to this, run the example with the dummy waveform first and print the shapes. Then try a real recording that you resampled to 16 kHz.

If you know the field, this is a self-supervised encoder, pretrained on 197k hours of English speech and general audio. It is meant as a base for tasks such as speech recognition and audio tagging. After fine-tuning, the card reports a LibriSpeech test-clean error rate of 1.6 and a test-other rate of 2.9.

16 kHz waveform goes in
Zipformer encoder processes it
Frame features come out
Fine-tune for ASR or tagging

Good to know: The card states results as the maker's own. After fine-tuning on AudioSet it reports 50.0 mAP on the full set and 39.4 on the balanced set.

Title: Audio tagging mAP after fine-tuning
AudioSet Balanced: 39.4
AudioSet Full: 50.0

• Remote code: loading uses the option that runs the repository's own code, so read it first
• Pretraining data: about 197k hours of unlabeled English speech and general audio
• License: Apache 2.0
• Cost to run: none named
• Data it reaches: the audio you feed it locally, plus files downloaded from Hugging Face
• Not for: out-of-the-box transcription or tagging, since it needs labeled fine-tuning

On Hugging Face, 13 November 2026

• Task: feature extraction
• Library: transformers
• Parameters: 597M
• Weights: about 2.4 GB in F32; about 298 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,668
• Likes: 10
• License: apache-2.0
• Tags: transformers, safetensors, spear, feature-extraction, huggingface, speech, audio, speech-representation-learning, audio-representation-learning, self-supervised-learning, automatic-speech-recognition, audio-tagging

Links

• Model: https://huggingface.co/marcoyang/spear-xlarge-speech-audio-v2
• Files: https://huggingface.co/marcoyang/spear-xlarge-speech-audio-v2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download marcoyang/spear-xlarge-speech-audio-v2

Then follow the usage in its model card. Nothing in this kit runs by itself.