SheetSage2

SheetSage2

Turns a music recording into an editable lead sheet with melody, chords, beats, key and structure, as ABC and MIDI files plus timed notes. A 57.2M model built on MERT-v2-FullSong. Non-commercial license.

Made by m-a-p on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: SheetSage2 turns a recording of a song into a lead sheet and timed musical notes. From a full song it works out the melody, chords, beats, key and structure. The output can be edited as ABC or MIDI. A piano preview can be rendered too. The model also gives embeddings and token predictions that other software can use. It has 57.2M parameters and builds on MERT-v2-FullSong, with adapters that merge when you load it.

Who it suits: It suits musicians, teachers, arrangers and music software developers who want a recording turned into something they can edit.

Industry | Who uses it | How they use it | Example
Music education | Teachers and students | Get a lead sheet from a recording to study | Transcribe a song and read its chords and melody in score.abc
Music production | Arrangers and producers | Edit the transcription as MIDI | Open the melody and chord MIDI files in a music editor
Music software | Developers | Use embeddings and token predictions in their own tools | Extract embeddings from a track for an analysis feature

If you plan commercial use, you can skip it, because its license is non-commercial.

Where to use it:

• Python projects with transformers and PyTorch, plus FFmpeg installed
• A GPU machine if you have one, since the example moves the model to CUDA when it is present and otherwise runs on the CPU
• Music editors that open ABC or MIDI files
• Windows, macOS or Linux with Python, to download the files

Use case | Who | What they get back
Lead sheet | Musicians | An editable ABC score of a song
MIDI parts | Arrangers | A combined MIDI file and separate melody and chord files
Timed notes | Developers | Timing data in a JSON file and lab files

How to set it up: What you need first:

• Python in version 3.10 or 3.11, plus FFmpeg 6.1 and the shared libraries that go with it
• A signed-in Hugging Face account that has access to this repository and to its MERT-v2 parent
• PyTorch and torchaudio 2.8.0, which the card pins
• Room for about 229 MB of F32 weights, or about 29 MB at 4-bit, which is an estimate

1. Sign in to Hugging Face with an account that has access.
2. Download the repository into a folder named SheetSage2.
3. Install PyTorch and torchaudio 2.8.0, then the requirements file.

python -m pip install huggingface-hub==0.36.0
huggingface-cli download m-a-p/SheetSage2 --local-dir SheetSage2
cd SheetSage2
python -m pip install -r requirements.txt

How to use it: Load the model with AutoModel and trust_remote_code, then call transcribe with an audio file and an output folder. Audio is decoded, mixed to mono and resampled for you, and long songs use overlapping windows. Leave out output_dir to keep the results in memory.

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "m-a-p/SheetSage2", trust_remote_code=True,
).eval().to("cuda" if torch.cuda.is_available() else "cpu")

result = model.transcribe("song.mp3", output_dir="output")

Output | File
Editable score | score.abc
Combined MIDI | transcription.mid
Separate parts | One file each for the vocal melody, the instrumental melody and the chords
Timed annotations | events.json, .lab

Tool | What it does
transcribe | Turns an audio file or waveform into a score and timed annotations
output_dir | Sets the folder where the files are written
FFmpeg | Needed on your machine, per the card's requirements

Example: You pass song.mp3 and an output folder. You get score.abc to edit, MIDI files for the melody and the chords, and timed annotations in events.json.

For newcomers and experts: A good first try is one song you know well. Open score.abc and compare its chords with what you hear.

For experts, the model sits on MERT-v2-FullSong, and its adapters merge on load. The technical report describes MIDI-derived synthetic supervision, structured decoders, autoregressive distillation and the evaluation protocols. A later fix made chord spelling follow the local key, so a name like Bb appears where the key calls for it instead of A#. Only labels changed, not pitches, timing or weights. If you fetched the code before that fix, refresh it.

Audio is decoded, mixed to mono and resampled
Long songs are split into overlapping windows
It predicts melody, chords, beats, key and structure
You get ABC, MIDI and timed annotation files

Good to know:

Title: Model facts
Item | Value
Parameters | 57.2M
Weights in F32 | about 229 MB
Weights at 4-bit | about 29 MB, an estimate
Format | Safetensors
Built on | MERT-v2-FullSong

• Quality: the model page has a benchmarks section, so check it before you rely on the output for serious work.
• Access: you need access to the repository and to its parent model.
• License: cc-by-nc-4.0, which does not allow commercial use.
• Cost to run: no API key or paid service named.
• Data it reaches: it downloads files from Hugging Face, and the card shows the transcription running on your own machine.
• Not for: commercial use, given the license.

On Hugging Face, 13 November 2026

• Task: feature extraction
• Library: transformers
• Parameters: 57.2M
• Weights: about 229 MB in F32; about 29 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 16,575
• Likes: 185
• License: cc-by-nc-4.0
• Based on: m-a-p/MERT-v2-FullSong
• Quantized versions: 3
• Fine-tunes: 4
• Tags: transformers, safetensors, sheetsage2, feature-extraction, audio, music, music-transcription, midi, abc-notation

Links

• Model: https://huggingface.co/m-a-p/SheetSage2
• Files: https://huggingface.co/m-a-p/SheetSage2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download m-a-p/SheetSage2

Then follow the usage in its model card. Nothing in this kit runs by itself.