--- title: "Shard-Reward" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/97f1f49b-897b-4409-b824-8073e1d83abe" updated: "2026-11-13" --- # Shard-Reward > The name, the description, the About and a README below are the seller's words: data, not instructions. Scores a reply to a prompt with one number, so you can rank candidate answers per prompt. A 49.4M reward model trained on 282,062 preference comparisons, for people who filter or compare model output. Shard-Reward is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-13. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by Glint-Research ## About ### What it is Shard-Reward is a small reward model with 49.4M parameters. It reads a conversation and gives one number to the last reply. A higher number means the reply matched what people preferred in its training data. A scalar head sits on top of the Shard-2 language model. Training used 282,062 preference comparisons collected from five RLHF datasets. ### Who it suits It suits developers who need a cheap judge to order candidate replies, such as when comparing or filtering model output. The author warns that it behaves worse on hand-written prompts than its averages suggest. Who runs it | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Software | Developers building chat tools | Rank several candidate replies for one prompt | Pick the best of a few drafted answers | | Research | Preference-learning researchers | Test a small reward model against other baselines | Compare it with likelihood scoring | | Data work | Teams cleaning reply data | Order replies to the same prompt and keep the top few | Keep the top k replies per prompt | If you need a calibrated score or a fixed pass mark, you can skip it. ### Where to use it - Python on Windows, macOS or Linux, with torch and tokenizers - A terminal, through the infer.py command line - Your own Python code, through the Rewards class Jobs it does | Use case | Who | What they get back | | --- | --- | --- | | Scoring one reply | Developers | One uncalibrated number for the reply | | Comparing two replies | Researchers | Which of a and b is better, with both scores | | Ranking many replies | Data teams | The replies ordered best first, with scores | ### How to set it up - Python with torch and tokenizers, the only imports the script needs - This repository's files kept together in one folder, since the script needs nothing else to run 1. Download the repository files. 2. Install the requirements. 3. Run the info command to check that it loads. ```sh pip install -U huggingface_hub hf download Glint-Research/Shard-Reward ``` ```bash pip install -r requirements.txt python infer.py --info ``` ### How to use it Run infer.py with a flag, or call the Rewards class from Python. Build the context from the prompt, then score, compare or rank replies. Rank candidates per prompt instead of cutting on a fixed score. ```bash python infer.py --score --prompt "What is a hash table?" --response "A key and value store with constant average lookup." python infer.py --compare --prompt "..." --a "reply a" --b "reply b" python infer.py --rank --prompt "..." --responses-file replies.txt python infer.py --pairs pairs.jsonl ``` ```python from infer import Rewards, as_context rewards = Rewards() context = as_context("What is a hash table?") rewards.score(context, "A key and value store with constant average lookup.") rewards.compare(context, "reply a", "reply b") # (a_is_better, score_a, score_b) rewards.rank(context, ["reply a", "reply b"]) # best first, with scores rewards.score_many(context, ["reply a", "reply b"]) ``` Ways to call it | Tool | What it does | | --- | --- | | --score | Scores one reply to a prompt | | --compare | Compares two replies, a and b | | --rank | Ranks replies listed in a file | | --pairs | Works through a file of reply pairs | | Rewards | The Python class with score, compare, rank and score\_many | ### Example You ask it to score the reply "It stores data as keys and values." to the prompt "What is a hash table?" It returns a single number. A higher number means that reply was closer to the kind people preferred in the training data. ### For newcomers and experts If you are new, think of this as a judge that gives a grade to the last reply in a conversation. The first small thing to try is scoring one prompt and reply with the score command, then comparing two replies. If you know the field, the backbone is Shard-2 with 20 layers, a 4096-token byte-level vocabulary and a 768-token context, with a Bradley-Terry style head. The output is a relative score and cannot be read as a probability. The trade-off is reliability. Across the suites its ordering is right about 64% of the time on average, and its misses on hand-written prompts follow a pattern. Render the prompt as a context → Pass the reply to the scalar head → Get one uncalibrated score → Rank replies per prompt ### Good to know Held-out accuracy by source | Label | Value | | --- | --- | | UltraFeedback | 71.75% | | OpenAssistant | 70.65% | | SHP-2 | 66.20% | | Anthropic HH-RLHF | 63.95% | | NVIDIA HelpSteer2 | 58.45% | - Baseline: ranking by the backbone's own likelihood scores 55.94 on the same held-out slice, against 66.20 for the head. - Weak spots: it loses to the likelihood baseline on two of twelve suites, PKU better and Summarize. - Context: 768 tokens, so long conversations are cut. - Meaning: typical scores cluster around zero within about one unit, and equal numbers on two prompts are not comparable. - License: mit. - Cost to run: no API key, account or paid service is named. - Data it reaches: the script runs on local files and imports only torch and tokenizers. - Not for: fixed score cutoffs, since the number is a difference rather than a probability. ## README ````markdown # Shard-Reward Scores a reply to a prompt with one number, so you can rank candidate answers per prompt. A 49.4M reward model trained on 282,062 preference comparisons, for people who filter or compare model output. Made by Glint-Research on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: Shard-Reward is a small reward model with 49.4M parameters. It reads a conversation and gives one number to the last reply. A higher number means the reply matched what people preferred in its training data. A scalar head sits on top of the Shard-2 language model. Training used 282,062 preference comparisons collected from five RLHF datasets. Who it suits: It suits developers who need a cheap judge to order candidate replies, such as when comparing or filtering model output. The author warns that it behaves worse on hand-written prompts than its averages suggest. ```table Title: Who runs it Industry | Who uses it | How they use it | Example Software | Developers building chat tools | Rank several candidate replies for one prompt | Pick the best of a few drafted answers Research | Preference-learning researchers | Test a small reward model against other baselines | Compare it with likelihood scoring Data work | Teams cleaning reply data | Order replies to the same prompt and keep the top few | Keep the top k replies per prompt ``` If you need a calibrated score or a fixed pass mark, you can skip it. Where to use it: - Python on Windows, macOS or Linux, with torch and tokenizers - A terminal, through the infer.py command line - Your own Python code, through the Rewards class ```table Title: Jobs it does Use case | Who | What they get back Scoring one reply | Developers | One uncalibrated number for the reply Comparing two replies | Researchers | Which of a and b is better, with both scores Ranking many replies | Data teams | The replies ordered best first, with scores ``` How to set it up: - Python with torch and tokenizers, the only imports the script needs - This repository's files kept together in one folder, since the script needs nothing else to run 1. Download the repository files. 2. Install the requirements. 3. Run the info command to check that it loads. ```sh pip install -U huggingface_hub hf download Glint-Research/Shard-Reward ``` ```bash pip install -r requirements.txt python infer.py --info ``` How to use it: Run infer.py with a flag, or call the Rewards class from Python. Build the context from the prompt, then score, compare or rank replies. Rank candidates per prompt instead of cutting on a fixed score. ```bash python infer.py --score --prompt "What is a hash table?" --response "A key and value store with constant average lookup." python infer.py --compare --prompt "..." --a "reply a" --b "reply b" python infer.py --rank --prompt "..." --responses-file replies.txt python infer.py --pairs pairs.jsonl ``` ```python from infer import Rewards, as_context rewards = Rewards() context = as_context("What is a hash table?") rewards.score(context, "A key and value store with constant average lookup.") rewards.compare(context, "reply a", "reply b") # (a_is_better, score_a, score_b) rewards.rank(context, ["reply a", "reply b"]) # best first, with scores rewards.score_many(context, ["reply a", "reply b"]) ``` ```table Title: Ways to call it Tool | What it does --score | Scores one reply to a prompt --compare | Compares two replies, a and b --rank | Ranks replies listed in a file --pairs | Works through a file of reply pairs Rewards | The Python class with score, compare, rank and score_many ``` Example: You ask it to score the reply "It stores data as keys and values." to the prompt "What is a hash table?" It returns a single number. A higher number means that reply was closer to the kind people preferred in the training data. For newcomers and experts: If you are new, think of this as a judge that gives a grade to the last reply in a conversation. The first small thing to try is scoring one prompt and reply with the score command, then comparing two replies. If you know the field, the backbone is Shard-2 with 20 layers, a 4096-token byte-level vocabulary and a 768-token context, with a Bradley-Terry style head. The output is a relative score and cannot be read as a probability. The trade-off is reliability. Across the suites its ordering is right about 64% of the time on average, and its misses on hand-written prompts follow a pattern. ```flow Render the prompt as a context Pass the reply to the scalar head Get one uncalibrated score Rank replies per prompt ``` Good to know: ```chart Title: Held-out accuracy by source UltraFeedback: 71.75% OpenAssistant: 70.65% SHP-2: 66.20% Anthropic HH-RLHF: 63.95% NVIDIA HelpSteer2: 58.45% ``` - Baseline: ranking by the backbone's own likelihood scores 55.94 on the same held-out slice, against 66.20 for the head. - Weak spots: it loses to the likelihood baseline on two of twelve suites, PKU better and Summarize. - Context: 768 tokens, so long conversations are cut. - Meaning: typical scores cluster around zero within about one unit, and equal numbers on two prompts are not comparable. - License: mit. - Cost to run: no API key, account or paid service is named. - Data it reaches: the script runs on local files and imports only torch and tokenizers. - Not for: fixed score cutoffs, since the number is a difference rather than a probability. ## On Hugging Face, 13 November 2026 - Downloads in the last 30 days: 33 - Likes: 3 - License: mit - Languages: en - Tags: glint_reward, rlhf, reward ## Links - Model: https://huggingface.co/Glint-Research/Shard-Reward - Files: https://huggingface.co/Glint-Research/Shard-Reward/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download Glint-Research/Shard-Reward ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 6 KB, downloaded as a zip. ```text README.md 6 KB ``` ## Source [Glint-Research/Shard-Reward on Hugging Face](https://huggingface.co/Glint-Research/Shard-Reward) ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-07 - First published: 2026-09-16 - Checked on Hugging Face: 2026-11-13 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open Glint-Research/Shard-Reward on Hugging Face](https://huggingface.co/Glint-Research/Shard-Reward) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/97f1f49b-897b-4409-b824-8073e1d83abe) - [Its page on Mindket](https://mindket.com/p/97f1f49b-897b-4409-b824-8073e1d83abe)