Shard-Reward

Shard-Reward

Scores a reply to a prompt with one number, so you can rank candidate answers per prompt. A 49.4M reward model trained on 282,062 preference comparisons, for people who filter or compare model output.

Made by Glint-Research on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Shard-Reward is a small reward model with 49.4M parameters. It reads a conversation and gives one number to the last reply. A higher number means the reply matched what people preferred in its training data. A scalar head sits on top of the Shard-2 language model. Training used 282,062 preference comparisons collected from five RLHF datasets.

Who it suits: It suits developers who need a cheap judge to order candidate replies, such as when comparing or filtering model output. The author warns that it behaves worse on hand-written prompts than its averages suggest.

Title: Who runs it
Industry | Who uses it | How they use it | Example
Software | Developers building chat tools | Rank several candidate replies for one prompt | Pick the best of a few drafted answers
Research | Preference-learning researchers | Test a small reward model against other baselines | Compare it with likelihood scoring
Data work | Teams cleaning reply data | Order replies to the same prompt and keep the top few | Keep the top k replies per prompt

If you need a calibrated score or a fixed pass mark, you can skip it.

Where to use it:

• Python on Windows, macOS or Linux, with torch and tokenizers
• A terminal, through the infer.py command line
• Your own Python code, through the Rewards class

Title: Jobs it does
Use case | Who | What they get back
Scoring one reply | Developers | One uncalibrated number for the reply
Comparing two replies | Researchers | Which of a and b is better, with both scores
Ranking many replies | Data teams | The replies ordered best first, with scores

How to set it up:

• Python with torch and tokenizers, the only imports the script needs
• This repository's files kept together in one folder, since the script needs nothing else to run

1. Download the repository files.
2. Install the requirements.
3. Run the info command to check that it loads.

pip install -U huggingface_hub
hf download Glint-Research/Shard-Reward

pip install -r requirements.txt

python infer.py --info

How to use it: Run infer.py with a flag, or call the Rewards class from Python. Build the context from the prompt, then score, compare or rank replies. Rank candidates per prompt instead of cutting on a fixed score.

python infer.py --score --prompt "What is a hash table?" --response "A key and value store with constant average lookup."
python infer.py --compare --prompt "..." --a "reply a" --b "reply b"
python infer.py --rank --prompt "..." --responses-file replies.txt
python infer.py --pairs pairs.jsonl

from infer import Rewards, as_context

rewards = Rewards()
context = as_context("What is a hash table?")

rewards.score(context, "A key and value store with constant average lookup.")
rewards.compare(context, "reply a", "reply b")     # (a_is_better, score_a, score_b)
rewards.rank(context, ["reply a", "reply b"])      # best first, with scores
rewards.score_many(context, ["reply a", "reply b"])

Title: Ways to call it
Tool | What it does
--score | Scores one reply to a prompt
--compare | Compares two replies, a and b
--rank | Ranks replies listed in a file
--pairs | Works through a file of reply pairs
Rewards | The Python class with score, compare, rank and score_many

Example: You ask it to score the reply "It stores data as keys and values." to the prompt "What is a hash table?" It returns a single number. A higher number means that reply was closer to the kind people preferred in the training data.

For newcomers and experts: If you are new, think of this as a judge that gives a grade to the last reply in a conversation. The first small thing to try is scoring one prompt and reply with the score command, then comparing two replies.

If you know the field, the backbone is Shard-2 with 20 layers, a 4096-token byte-level vocabulary and a 768-token context, with a Bradley-Terry style head. The output is a relative score and cannot be read as a probability. The trade-off is reliability. Across the suites its ordering is right about 64% of the time on average, and its misses on hand-written prompts follow a pattern.

Render the prompt as a context
Pass the reply to the scalar head
Get one uncalibrated score
Rank replies per prompt

Good to know:

Title: Held-out accuracy by source
UltraFeedback: 71.75%
OpenAssistant: 70.65%
SHP-2: 66.20%
Anthropic HH-RLHF: 63.95%
NVIDIA HelpSteer2: 58.45%

• Baseline: ranking by the backbone's own likelihood scores 55.94 on the same held-out slice, against 66.20 for the head.
• Weak spots: it loses to the likelihood baseline on two of twelve suites, PKU better and Summarize.
• Context: 768 tokens, so long conversations are cut.
• Meaning: typical scores cluster around zero within about one unit, and equal numbers on two prompts are not comparable.
• License: mit.
• Cost to run: no API key, account or paid service is named.
• Data it reaches: the script runs on local files and imports only torch and tokenizers.
• Not for: fixed score cutoffs, since the number is a difference rather than a probability.

On Hugging Face, 13 November 2026

• Downloads in the last 30 days: 33
• Likes: 3
• License: mit
• Languages: en
• Tags: glint_reward, rlhf, reward

Links

• Model: https://huggingface.co/Glint-Research/Shard-Reward
• Files: https://huggingface.co/Glint-Research/Shard-Reward/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Glint-Research/Shard-Reward

Then follow the usage in its model card. Nothing in this kit runs by itself.