getting-over-it-rl-jax

getting-over-it-rl-jax

Plays the full game Getting Over It from the bottom to the end with a small trained policy network that learned inside a JAX physics simulator. For reinforcement learning researchers and curious tinkerers.

Made by zimmerk4 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This project holds one small policy network that finishes the game Getting Over It with Bennett Foddy in a single climb, with no saving or loading. It learned only inside a JAX copy of the game's physics and rules, then drove the real game through a BepInEx plugin without changes.

Who it suits: Reinforcement learning researchers who want a worked example of training in a fast simulator and transferring to the real thing, plus people curious how a bot beats a famously hard game.

Industry | Who uses it | How they use it | Example
AI research | RL researchers | Study simulator-to-real transfer with a released policy | Load the weights and play the real game through the plugin
Education | Students | Read a compact GPU training setup in JAX | Follow how starts, rewards and wall-by-wall training were organised
Game AI | Hobbyists | Watch a learned bot climb a hard game | Compare its win rate and times with their own runs
Software | JAX developers | Reuse the numpy-only way to compute an action from saved arrays | Run the policy without JAX installed

You can skip it if you want a general game-playing agent, since the weights were trained for this one game.

Where to use it: The files download on Windows, macOS or Linux with Python, and the policy runs wherever numpy runs.

• Python with numpy, for computing actions from the saved weights
• The maker's code repository, whose goi_jax.play loads the weights directly
• The real game with its BepInEx plugin, for the full climb

Use case | Who | What they get back
Real-game play | Hobbyists | A climb from the bottom to the end, with a measured win rate
Policy inspection | Researchers | Plain arrays for the network, normaliser and training config
Result checking | Reviewers | The log and report files behind the 20-climb evaluation

How to set it up: You need Python with pip for the files. Playing the real game also needs the maker's code repository, the game and its plugin installed.

• Python with pip
• numpy, to run the short action snippet
• The code repository on its jax-engine branch, plus the game and BepInEx plugin, for real-game play

1. Install the Hugging Face download tool and fetch the files.
2. Get the maker's code repository and switch to its jax-engine branch.
3. Install the game and the plugin as that repository describes.
4. Run the play command from the repository.

pip install -U huggingface_hub
hf download zimmerk4/getting-over-it-rl-jax

How to use it: Point the play module at policy.npz, which loads the network directly. You can also compute one action yourself with numpy from the saved arrays, as the second block shows.

python -m goi_jax.play policy.npz --backend real --no-ends --steps 9000

import numpy as np
w = np.load("policy.npz")
x = np.clip((obs - w["obs_mean"]) / np.sqrt(w["obs_var"] + 1e-8), -10, 10)
h = np.tanh(np.tanh(x @ w["pi0_w"] + w["pi0_b"]) @ w["pi1_w"] + w["pi1_b"])
mean = np.tanh(h @ w["mu_w"] + w["mu_b"])
action = np.clip(mean + 0.5 * np.exp(w["log_std"]) * np.random.randn(2), -1, 1)   # play.py's default noise 0.5
mouse = action * 12.0

Tool | What it does
policy.npz | The saved network, with its input scaling and the settings it was trained under
config.json | Holds the training configuration of the final run
eval | The material behind the 20 timed climbs
media/jax-win.gif | Shows a 97-second winning climb at double speed
goi_jax.play | Loads policy.npz and plays the game

Example: Run the play command with policy.npz against the real game. The network starts at the bottom and steers the hammer with mouse movements until it reaches the end. In the maker's 20 timed climbs it won 17, with a median of 1 minute 55 seconds of play.

For newcomers and experts: A policy network is a small program that looks at the game state and picks the next mouse movement. This one was trained by trial and error in a fast copy of the game, then used on the real game. A first small try is to download the files and read the numpy snippet above.

For experts, the network takes 105 numbers describing the player, the terrain ahead and the recent history, and returns a 2-D mouse movement scaled by 12. Both the policy and the value network are 256 by 256 tanh MLPs. Training used GPU-side PPO in the PureJaxRL style, with starts spread along the route and from Go-Explore. The trade is fidelity against speed, since the card reports the simulator matching recordings of the real game while running about 45,000 to 50,000 steps a second.

Reimplement the game's physics in JAX
Train with PPO on many parallel games
Train copies from just below each wall
Save the final network as policy.npz
Play the real game through the plugin

Good to know: The win rate comes from the maker's own 20 real-game climbs, each given up to 5 minutes.

• Not every climb finishes: 3 of the 20 hit the 5-minute limit, and the highest of those reached 413 m before a fall.
• Safe loading: The weights are plain arrays that load without pickle.
• Training scale: the last copy ran 2,512 updates, and all its ancestors together used roughly 1.3 billion simulated steps.

Title: Real-game climbs out of 20
Won: 17 climbs
Did not finish in 5 minutes: 3 climbs

• License: MIT.
• Cost to run: none named beyond the game itself and its BepInEx plugin, which real-game play needs.
• Data it reaches: the weights and logs in the download, plus the game running on your computer through a plugin.
• Not for: other games, since the card describes training for this one game only.

On Hugging Face, 14 November 2026

• Task: reinforcement learning
• Library: jax
• Downloads in the last 30 days: 87
• Likes: 5
• License: mit
• Tags: reinforcement-learning, sim-to-real, getting-over-it

Links

• Model: https://huggingface.co/zimmerk4/getting-over-it-rl-jax
• Files: https://huggingface.co/zimmerk4/getting-over-it-rl-jax/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download zimmerk4/getting-over-it-rl-jax

Then follow the usage in its model card. Nothing in this kit runs by itself.