gemma-4-E4B-it-weight-space-programming

gemma-4-E4B-it-weight-space-programming

Google's Gemma 4 E4B-it with 0.002% of its weights edited directly, no training, to fix stalls in thinking mode with tool calls and multi-step math. Also ships as a small patch file.

Made by jstarks on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds Gemma 4 E4B-it with a direct edit to 64 columns in MLP layers 38 to 41, about 163,840 parameters or 0.00205% of the model. No gradient descent, backpropagation or fine-tuning was used.

The edit targets two problems the author describes in thinking mode. The model either rambles in long outlines instead of calling a tool, or burns its token budget on meta-commentary before it does the math.

Who it suits: People who run Gemma 4 E4B-it with thinking mode on and want it to call tools more reliably, and researchers curious about editing weights directly instead of training.

Industry | Who uses it | How they use it | Example
Software | Agent builders | Make a small local model call tools in thinking mode | A web-search helper on a desktop GPU
Education | Math tutoring developers | Get multi-step solutions inside a short token budget | A homework helper that starts the algebra directly
Research | Interpretability researchers | Study weight-space edits | Reading the card's layer-by-layer findings

You can skip it if you do not use thinking mode, since the card's tests are about that mode, or if you need results from large benchmark suites.

Where to use it: It loads through the transformers library, and a loader script handles smaller GPUs. The author tested on an RTX 4070 Ti with 12 GB.

• Python with transformers and PyTorch
• A 12 GB GPU using the included model_loader.py script
• Any existing gemma-4-E4B-it checkpoint, with the small patch file applied in memory

Use case | Who | What they get back
Tool calling with thinking on | Agent builders | More valid tool calls in the card's 8-query test
Short-budget math | Developers | More problems solved within 420 tokens
Quick patching | Hobbyists | An edit applied to a local checkpoint in under a second

How to set it up: You need Python and either the full 16 GB model or an existing Gemma 4 E4B-it checkpoint plus the 641 KB patch file. The card gives two ways in.

• Python with pip, transformers and PyTorch
• Option A: the full 16 GB bfloat16 model from this repository
• Option B: an existing gemma-4-E4B-it checkpoint, and the patch file from this repository

1. Install Python, transformers and PyTorch.
2. Download the repository files, or only the patch if you already hold the base model.
3. Run one of the loading snippets below.

pip install -U huggingface_hub
hf download jstarks/gemma-4-E4B-it-weight-space-programming

How to use it: Option A loads the whole edited model with transformers. Option B patches a base checkpoint in memory.

from transformers import AutoProcessor, AutoModelForMultimodalLM
import torch

model_id = "jstarks/gemma-4-E4B-it-weight-space-programming"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
    model_id, device_map="auto", torch_dtype=torch.bfloat16
)

On a 12 GB GPU, the included loader keeps the 42 active layers on the GPU and moves the large per-layer embedding table to the CPU.

from model_loader import load_gemma4_e4b_text

model, tokenizer = load_gemma4_e4b_text("jstarks/gemma-4-E4B-it-weight-space-programming")

from apply_patch import apply_gemma4_weight_space_patch
from model_loader import load_gemma4_e4b_text

# 1. Load your existing Gemma-4-E4B-it checkpoint
model, tokenizer = load_gemma4_e4b_text("unsloth/gemma-4-E4B-it")

# 2. Apply the 64-column weight-space patch (163,840 parameters across Layers 38-41)
apply_gemma4_weight_space_patch(model, "gemma4_e4b_weight_space_patch.safetensors")

Example: You already have unsloth's gemma-4-E4B-it on disk. You load it with the loader, apply the 641 KB patch, and the card says the edited model is ready in under a second without a 16 GB download.

For newcomers and experts: If you are new, this is a slightly modified Gemma. Load it with Option A, switch thinking on, give it a tool such as web search, and see whether it calls the tool.

If you know the field, the edit changes RMSNorm-preconditioned columns in layers 38 to 41 and was inspired by a paper on weight-space programming. The card also describes layer 23 as a boundary hub where key-value states start to be shared.

Start from gemma-4-E4B-it
Edit 64 MLP columns in layers 38 to 41
Save as a full model or a 641 KB patch
Load in transformers or patch in memory

Good to know: The test suites are small, with 8 queries, 6 math problems and 3 end-to-end runs, so treat the numbers as a demo. The card reports these before and after results.

Title: Success rate before and after the edit, percent
Tool calls before: 0.0 percent
Tool calls after: 75.0 percent
Math solved before: 33.3 percent
Math solved after: 66.7 percent
Live agent runs before: 0.0 percent
Live agent runs after: 66.7 percent

• License: Gemma license, the one shown on the page
• Cost to run: none named, but a GPU with about 12 GB of memory was used in the tests
• Data it reaches: the model files you download, and the prompts and tool results you give it
• Not for: use without thinking mode in mind, since the tests cover thinking mode only

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 8B
• Weights: about 16 GB in BF16; about 4 GB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 125
• Likes: 3
• License: gemma
• Based on: google/gemma-4-E4B-it
• Tags: transformers, safetensors, gemma4, image-text-to-text, weight-space-programming, mechanistic-interpretability, tool-calling, agentic, zero-training, conversational

Links

• Model: https://huggingface.co/jstarks/gemma-4-E4B-it-weight-space-programming
• Files: https://huggingface.co/jstarks/gemma-4-E4B-it-weight-space-programming/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download jstarks/gemma-4-E4B-it-weight-space-programming

Then follow the usage in its model card. Nothing in this kit runs by itself.