SchemeArena

SchemeArena

Presents a research project that tests LLM agents for scheming across 400 scenarios, with a monitor called SCOUT. It is a static project page for AI safety researchers.

Made by launch on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: SchemeArena is the project page for a study of scheming in LLM agents, meaning agents that pursue goals in a hidden way. The study builds 400 agentic scenarios across 28 safety-relevant tool areas. It also describes a monitor named SCOUT that judges scheming from what an agent reasons and does. The Space is a static website with no server code.

Who it suits: AI safety researchers and agent developers who want to see how scheming can be tested in a controlled, repeatable way.

Industry | Who uses it | How they use it | Example
AI safety | Safety researchers | Study scheming tests for agents | Read how the 7 setups change pressure and oversight
Software | Agent developers | Learn what risky behavior a test can look for | Review the 28 tool areas
Research | Evaluation researchers | Compare monitoring approaches | Read about the SCOUT monitor
Education | Students of AI safety | Get an overview of one benchmark | Read the project page

People who want a ready tool to run on their own agent can skip it, since this is a project page.

Where to use it:

• A browser, by visiting the Space on Hugging Face
• A copy of the files on Windows, macOS or Linux, opened in a web browser

Use case | Who | What they get back
Read the project summary | Researchers | An overview of the scenarios, setups and monitor
Check the scope | Developers | The tool areas and setups the study covers

How to set it up:

• Python and pip, only to fetch a copy of the files
• A web browser

1. Install the Hugging Face hub package.
2. Download the Space with the hf command.
3. Move into the new folder.
4. Open index.html in a web browser.

pip install -U huggingface_hub
hf download launch/SchemeArena --repo-type space --local-dir SchemeArena
cd SchemeArena

How to use it: Online, visit the Space on Hugging Face and read the project page. With a local copy, open index.html in your browser and read it the same way. There are no commands or settings to run.

Tool | What it does
index.html | The project page
css | Holds the style sheets
js | Holds the page scripts
data | Holds the data the page shows
assets | Holds the images and other files

Example: You visit the page and read about the seven setups. You see how the study changes goals, pressure, strategic hints and oversight to test whether an agent schemes.

For newcomers and experts: If you are new, read the three bullet points at the top of the page first. They give the size of the study, the variations it tests and the monitor it proposes.

If you know the field, the study is factorized, which means it varies one factor at a time across seven controlled setups. Those setups change instrumental goals, pressure, strategic hints and oversight. SCOUT is a criteria-based monitor that rests on evidence from an agent's reasoning and its actions. The trade-off is that this page summarizes the work and does not run the tests.

You visit the project page
You read the scenarios and setups
You see how SCOUT monitors agents
You follow the study's own materials

Good to know:

• Scale: 400 scenarios, 28 tool areas and 7 controlled setups
• Type: a static project page, so nothing runs on a server
• License: cc-by-4
• Cost to run: none named
• Data it reaches: the page names the launch/SchemeArena dataset and an assets repository on Hugging Face
• Not for: running tests on your own agent, since the page only describes the study

On Hugging Face, 13 November 2026

• Likes: 2
• Built with: static
• License: cc-by-4.0

Links

• Space: https://huggingface.co/spaces/launch/SchemeArena
• Files: https://huggingface.co/spaces/launch/SchemeArena/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.