ParsBench Leaderboard

ParsBench Leaderboard

A leaderboard from ParsBench that scores models on Persian language datasets, with the code to run or adapt your own copy. For Persian NLP researchers and engineers comparing models.

Made by ParsBench on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: This Leaderboard is a results page from ParsBench. Going by the datasets it names, it scores models on Persian language tasks. Those datasets include pn_summary, the ParsiNLU sets for entailment, reading comprehension and translation in both directions, and khayyam-challenge. The repository also holds the code that builds the board.

Who it suits: It suits people who work with Persian text and want measured scores before they pick a model.

Title: Who might use this leaderboard
Industry | Who uses it | How they use it | Example
Persian language technology | Researchers | Compare models on shared Persian datasets | Look up how a model scores on a named dataset
AI engineering | Engineers | Shortlist a model for a Persian feature | Compare two candidates on the task closest to theirs
Leaderboard hosting | Maintainers | Adapt the code to run a leaderboard of their own | Change the tasks in src/about.py

If you only work in other languages, you can skip it.

Where to use it: You can read it online or run a copy yourself.

• Web browser: the Space's page on Hugging Face, with nothing to install
• Windows, macOS or Linux: a local copy that runs with Python
• Hugging Face Spaces: where the hosted board runs, on the cpu-basic hardware tier

Title: What the leaderboard is good for
Use case | Who | What they get back
Compare models on Persian tasks | Engineers | Scores for each model across the named datasets
Study one task | Researchers | Results for a single dataset such as a translation pair
Run your own leaderboard | Maintainers | A local copy to adapt, fed by results saved as JSON files

How to set it up: The hosted board needs nothing. A local copy needs Python and pip.

• Python, with pip
• The Python packages that requirements.txt lists

1. Install the huggingface_hub package.
2. Download the Space's files into a folder.
3. Move into that folder.
4. Install Gradio 5.5.0.
5. Install the packages from requirements.txt.
6. Start the app with Python.

pip install -U huggingface_hub
hf download ParsBench/leaderboard --repo-type space --local-dir leaderboard
cd leaderboard
pip install gradio==5.5.0
pip install -r requirements.txt
python app.py

To point a copy at your own leaderboard, change the path in src/env.py and the tasks in src/about.py.

How to use it: Open the Space online, or start your copy with the command below. Then read the leaderboard table and compare the models you care about.

python app.py

Results go in as JSON files with a config part and a results part, in this shape.

{
    "config": {
        "model_dtype": "torch.float16", # or torch.bfloat16 or 8bit or 4bit
        "model_name": "path of the model on the hub: org/model",
        "model_sha": "revision on the hub",
    },
    "results": {
        "task_name": {
            "metric_name": score,
        },
        "task_name2": {
            "metric_name": score,
        }
    }
}

Title: The files that shape the board
Tool | What it does
src/env.py | Holds most of the variables you change, such as the leaderboard path
src/about.py | Holds the tasks
src/display/utils.py | Defines the main table's column names and properties
src/leaderboard/read_evals.py | Reads results and request files and turns them into table rows
src/submission/submit.py | Decides which submissions are allowed or filtered

Example: You need a model for Persian summaries. You open the leaderboard, look at the scores on the pn_summary dataset, and compare the models that rank near the top there.

For newcomers and experts: If you are new to this, open the Space and look at the main table before you change anything. Pick two models and compare their scores.

If you know the field, results are plain JSON with a config block that records the model's dtype, name and revision, and a results block of task and metric scores. The code reads those files, along with request files it creates itself, and turns them into dataframe rows. Submission checks live in src/submission/check_validity.py.

Results and request files are stored
The app reads and converts them
They become rows in the main table
You compare models on the board

Good to know: The board creates some folders of its own while it runs.

• Created folders: the app makes four working folders for queues and results, plus backups, and restarting the Space clears them if something goes wrong
• Requests: request files are made automatically by the tool
• Hosting: the hosted copy runs on cpu-basic hardware
• License: apache-2
• Cost to run: none named
• Data it reaches: the results and request files it reads, plus the Python packages pip installs from requirements.txt

On Hugging Face, 13 November 2026

• Likes: 40
• Built with: gradio
• License: apache-2.0

Links

• Space: https://huggingface.co/spaces/ParsBench/leaderboard
• Files: https://huggingface.co/spaces/ParsBench/leaderboard/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.