RL Environments on Hugging Face

RL Environments on Hugging Face

Lets you browse reinforcement learning environments on the Hugging Face Hub, look at their tasks and rewards, and run agent rollouts to compare results. It suits RL and agent researchers.

Made by FineEnvs on Hugging Face. These notes are Mindket's, written from its Space page and README; the app's own files and license are on Hugging Face.

About

What it is: This Space is the HF RL Explorer, a web app for locating RL environments that are hosted on the Hub. It covers public datasets and checked environment Spaces from OpenEnv, Harbor, MiMo, NeMo Gym and Verifiers. You can open a single task to see which tools and rewards it uses, then start agent runs on the supported frameworks and compare how they scored.

Who it suits: It suits people who build or test agents and want to compare environments, tasks and graders in one place.

Industry | Who uses it | How they use it | Example
AI research | RL researchers | Browse environments and read their tasks and rewards | Check how a task is graded before using it
Agent development | Agent builders | Run an agent on a task and see the grade | Run a Harbor task with one of the supported agents
Model evaluation | Evaluators | Compare rollouts by model | Look through public graded rollouts from the community

You can skip it if you do not work with agents or reinforcement learning environments.

Where to use it: You can use it in these places:

• A browser: open the Space's page on Hugging Face and nothing needs installing.
• Windows, macOS or Linux: build and run your own copy with Docker.
• Coding agents: OpenEnv Spaces are also reachable over MCP.

Use case | Who | What they get back
Finding environments | Researchers | An indexed view of datasets and environment Spaces
Running rollouts | Agent builders | Graded runs of an agent on a task
Comparing models | Evaluators | Community rollouts compared by model

How to set it up: You need these first for a local copy:

• Python and pip, to download the files
• Docker, to build and run the image
• A Hugging Face account, to run rollouts

The online page can be browsed without installing anything. For your own copy, follow the steps below.

1. Install the Hugging Face tools.
2. Download the Space's files into a folder.
3. Move into that folder.
4. Build the Docker image.
5. Run it with port 7860 open.

pip install -U huggingface_hub
hf download FineEnvs/RL-Explorer --repo-type space --local-dir RL-Explorer
cd RL-Explorer
docker build -t rl-explorer .
docker run -it -p 7860:7860 rl-explorer

How to use it: Open the Space and browse the datasets and environment Spaces it indexes. Open a task to read its instructions, tools and reward setup, then start a rollout from your Hugging Face account.

docker run -it -p 7860:7860 rl-explorer

Tool | What it does
Harbor datasets | Lists the tasks of each dataset. You pick an agent for a task, for example Pi, Terminus 2, mini-SWE-agent or OpenCode
OpenEnv Spaces | Shows what a server offers, with a playground, its files and MCP for coding agents
NeMo Gym, Verifiers, verl and SkyRL datasets | Shows each row as its framework reads it, with the commands to run it
MiMo-V2.6 RL | Provides the tasks of that release, run with their own tooling or in their Harbor form
Graders that call a model | Send the judge call through a relay, so the sandbox never holds your key
Community | Lists public graded rollouts from everyone, compared by model

Example: You open a Harbor task, read its instruction and choose one of the supported agents. The rollout runs in a sandbox on your Hugging Face account, and the task's own tests grade it.

For newcomers and experts: If you are new to this, open the Space and browse a few tasks before running anything. Reading a task's instruction and how it is graded shows what a reinforcement learning environment is.

If you know the field, the app indexes several frameworks and shows each row the way its framework means it. Rollouts use a sandbox billed by the hour and model tokens at the provider's price. Expected answers stay hidden. Files that hold solutions, reference patches and data kept beside a grader appear in listings, but their contents are not shown, over MCP too.

Browse datasets and Spaces
Open a task and read its setup
Sign in and start a rollout
The sandbox runs the agent
The task's grader scores it

Good to know: Rollouts run on your own Hugging Face account. Signing in asks for three permissions, inference-api for the model, jobs for the sandbox and read-repos for your private datasets, which only you see.

• License: apache-2
• Cost to run: hourly sandbox time plus model token charges set by the provider, or an endpoint of your own that speaks the OpenAI format
• Data it reaches: your Hugging Face account, Inference Providers and your private datasets, shown only to you
• Not for: seeing expected answers, which are withheld on every page and over MCP

On Hugging Face, 13 November 2026

• Likes: 40
• Built with: docker
• License: apache-2.0

Links

• Space: https://huggingface.co/spaces/FineEnvs/RL-Explorer
• Files: https://huggingface.co/spaces/FineEnvs/RL-Explorer/tree/main

Try it

Open the Space in your browser, or copy it to your own Hugging Face account from its page. Nothing in this kit runs by itself.