S1MB Leaderboard

S1MB Leaderboard

Compares System One Decision Models across more than 100 benchmarks, in a viewer of saved results. For researchers who want to see where a model does well or falls short.

Made by hotchpotch on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: S1MB Leaderboard is a viewer of saved benchmark results for System One Decision Models, which read text and instructions and return a structured decision. It compares them across more than 100 benchmarks and shows each benchmark on its own. It only displays measurements saved earlier, so no model gets loaded or executed here.

Who it suits: It suits researchers and model builders who want to compare decision models task by task and see where each one is strong or weak.

Industry | Who uses it | How they use it | Example
AI research | Researchers | Compare decision models across many benchmarks | Open one benchmark and read the saved results
Model development | Model builders | See where their model performs well or struggles | Look up a published model on the leaderboard
Software | Developers | Run the viewer locally in Docker | Build the image and open port 7860

If you want to test a model yourself, you can skip it, because the page only shows saved results.

Where to use it:

• A web browser, on the Space's page on Hugging Face
• Windows, macOS or Linux with Docker, from a local build of the Space's files

Use case | Who | What they get back
Compare models | Researchers | Results side by side across tasks
Explore one benchmark | Model builders | Saved measurements for a single benchmark
Switch evaluation scope | Anyone | English, Japanese or vision results, using the scope selector

How to set it up:

• A web browser, which is all the online page needs
• Docker and Python with pip, for a local copy

1. Open the Space's page on Hugging Face to use it online. Nothing needs installing.
2. For a local copy, install the Hugging Face hub package and download the Space's files.
3. Move into the downloaded folder.
4. Build the Docker image, then run it with port 7860 published.

pip install -U huggingface_hub
hf download hotchpotch/S1MB-leaderboard --repo-type space --local-dir S1MB-leaderboard
cd S1MB-leaderboard
docker build -t s1mb-leaderboard .
docker run -it -p 7860:7860 s1mb-leaderboard

How to use it: Open the page, or port 7860 on your machine once the container runs, and use the evaluation scope selector to switch between the English, Japanese and vision sets. Then compare models or open a single benchmark. The benchmark covers three kinds of decision.

Tool | What it does
Choice | Picks one option from a set of alternatives
Noul | Gives a yes or no judgment from the supplied definition
Score | Gives a number based on the supplied criteria

Example: Switch the scope to the Japanese set and look at how one model ranks across its tasks. Then open a single benchmark to see where that model does well and where it struggles.

For newcomers and experts: Open the Space page and pick the English scope first. Missing or failed measurements stay visibly incomplete, so a gap means no result and not a zero.

The tasks mix public NLP datasets with synthetic ones that vary the instructions, contexts and decision criteria. In the vision scope, Noul, Choice and Score are ranked, while the optional Box and gen-box tasks are shown apart and left out of the ranking. The scores are a reference for these tasks only, and they do not show how a model handles unseen tasks or whether its training data overlaps.

A model's results are published as a dataset
Display data is prepared from the results
The Docker image bundles the display data
The viewer shows it by scope

Good to know: A new result shows up once the display data is rebuilt and the Space goes out again.

• License: MIT for the viewer's code, while evaluation datasets and model checkpoints keep their own licenses
• Cost to run: none named
• Data it reaches: the display data bundled in its own Docker image
• Not for: running a model or checking whether a model has seen a task before

On Hugging Face, 13 November 2026

• Likes: 86
• Built with: docker
• License: mit

Links

• Space: https://huggingface.co/spaces/hotchpotch/S1MB-leaderboard
• Files: https://huggingface.co/spaces/hotchpotch/S1MB-leaderboard/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.