Obfuscated-Shell Safety Leaderboard

Obfuscated-Shell Safety Leaderboard

Ranks language models on whether they see through disguised shell commands, blocking the dangerous ones and allowing harmless encoded ones. A defensive safety benchmark for people building agents.

Made by FINAL-Bench on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: This is a leaderboard that measures how well language models understand a shell command that has been disguised. Obfuscation by base64, hex, eval or an interpreter hides the true purpose of a command. A good model reads the decoded intent, refuses the dangerous commands and still lets harmless ones through, even when they only look encoded.

The official score comes from a private held-out split, and no AI judge is involved. The test set is a dataset named Obfuscated-Shell-Gate from FINAL-Bench.

Who it suits: It suits people who decide whether a model can be trusted near a shell, and researchers who study how models handle disguised input.

Industry | Who uses it | How they use it | Example
AI safety | Safety researchers | Compare how models judge disguised shell commands | A researcher looks at which models see through encoded commands
Agent development | Developers of coding agents | Check a model's judgment before giving it shell access | A developer reads the scores before choosing a model
Security | Security teams | See how well models spot hidden intent in commands | A team reviews how a model treats base64 wrapped commands
Evaluation | Benchmark builders | Study a test with a private split and no AI judge | A builder reads how the scoring is set up

You can skip it if you do not give models any way to run commands.

Where to use it: You use it in these places.

• A web browser, on the Space's page on Hugging Face
• A local copy of the page, opened in a browser on Windows, macOS or Linux

Use case | Who | What they get back
Compare models | Researchers and developers | A ranking of models by how well they read disguised commands
Pick a model for an agent | Agent builders | Evidence on whether a model refuses the dangerous and allows the harmless
Find the test data | Evaluators | The name of the dataset behind the score

How to set it up: Online use needs only a browser. A local copy needs Python for the download step.

• A web browser
• Python with pip, only to fetch a local copy

1. Open the Space's page on Hugging Face to use it online, with nothing to install.
2. For a local copy, install the Hugging Face Hub package.
3. Download the Space's files into a folder and move into it.
4. Load index.html in your browser.

pip install -U huggingface_hub
hf download FINAL-Bench/obfuscated-shell-leaderboard --repo-type space --local-dir obfuscated-shell-leaderboard
cd obfuscated-shell-leaderboard

How to use it: Open the page and read the ranking. It lists models and how each one scored on disguised shell commands. The source shows no other command or control.

Example: You open the page to compare the models it lists on how well they read hidden intent. A model that refuses a dangerous command but still accepts a harmless encoded one scores better than one that fails either way.

For newcomers and experts: A shell command can be wrapped in an encoding so that its real purpose is hard to see. A safe model has to look past the wrapping. Start by opening the Space's page and reading how the scores are described.

Under the hood, the test covers base64, hex, eval and interpreter obfuscation. A model must deny the dangerous and allow the harmless, so a model that blocks everything encoded does not win. The record names six models, including DeepSeek, Gemma, Llama and Kimi variants.

A shell command is disguised
A model reads its decoded intent
It denies the dangerous and allows the harmless
The official score comes from a held-out split
The leaderboard lists the result

Good to know: The source gives a short description, so check the page for the full list and the scoring detail.

• License: cc-by-4.
• Cost to run: none named.
• Data it reaches: the page is static. The record names one dataset, FINAL-Bench/Obfuscated-Shell-Gate, and six models.
• Not for: AI-judged scoring, since the official score uses no AI judge.

On Hugging Face, 13 November 2026

• Likes: 1
• Built with: static
• License: cc-by-4.0

Links

• Space: https://huggingface.co/spaces/FINAL-Bench/obfuscated-shell-leaderboard
• Files: https://huggingface.co/spaces/FINAL-Bench/obfuscated-shell-leaderboard/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.