Qwen3.5-0.8B-weight-space-programming by ML-Intern-lab

Qwen3.5-0.8B-weight-space-programming by ML-Intern-lab

Solves math problems step by step with a small Qwen3.5-0.8B model whose weights an LLM edited directly, with no training, to stop repetition loops. For researchers and tinkerers testing weight edits.

Made by ML-Intern-lab on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is Qwen3.5-0.8B with 131,072 of its weights changed directly, with no training. A large language model called Astra read the weights, spotted a repetition loop, and rewrote 64 output columns in each of two layers. The original model keeps repeating itself on math problems until it hits the token limit, and this edited copy mostly avoids that.

Who it suits: This suits researchers who study how model behavior changes when weights are edited by hand, and developers who want a small math-reasoning model to experiment with.

Industry | Who uses it | How they use it | Example
Research | Model researchers | Study how a direct weight edit changes behavior | Compare the base and edited models on repetition loops
Education | Machine learning teachers | Show that behavior can change without training | Run both models on one math problem side by side
Software | Engineers testing small models | Try a small model for step-by-step math | Run the card's example prompt with vLLM

Skip it if you need a model for anything other than math, since the card reports results only on a math test.

Where to use it:

• Python with the transformers library
• vLLM, which the card uses to run and test it
• Windows, macOS or Linux, where you can download the files with Python
• A live demo Space on Hugging Face, where you can watch the base and edited models side by side

Use case | Who | What they get back
Step-by-step math answers | Students and developers | A worked solution that ends with a final answer in a box
Loop comparison | Researchers | Output from the base and edited models on the same problem
Weight-edit study | Researchers | A checkpoint whose edit the author published as an exact diff

How to set it up: You need these first:

• Python
• The huggingface_hub package
• vLLM and transformers, if you want to run the card's example

1. Install the Hugging Face hub package.
2. Download the model files to your computer. They take about 1.7 GB in BF16.
3. Install vLLM and transformers if you plan to run the example below.

pip install -U huggingface_hub
hf download ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming

How to use it: Load the model in vLLM, build the chat prompt with thinking switched on, and generate with the sampling settings the card lists. The card gives this script.

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer

ckpt = "akhaliq/Qwen3.5-0.8B-weight-space-programming"
tok = AutoTokenizer.from_pretrained(ckpt)
llm = LLM(model=ckpt, dtype="bfloat16", max_model_len=34816)

prompt = tok.apply_chat_template(
    [{"role": "user", "content": "What is $2^{10} + 24$?\nPlease reason step by step, and put your final answer within \\boxed{}."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=True)
sp = SamplingParams(n=1, temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5,
                    max_tokens=32768, seed=0)
print(llm.generate([prompt], sp)[0].outputs[0].text)

The script names the checkpoint under the akhaliq account, so check which id you load.

Tool | What it does
AutoTokenizer | Loads the tokenizer and turns your message into a chat prompt with thinking on
LLM | Loads the model in vLLM in bfloat16 with a maximum length of 34816
SamplingParams | Sets the sampling options, including a limit of 32768 new tokens

Example: You ask what 2 to the power of 10 plus 24 is, and tell the model to reason step by step and put the final answer in a box. You should get its step-by-step reasoning, ending with the answer inside a box.

For newcomers and experts: A model is a large set of numbers called weights, and people usually change them by training. Here an AI changed a small share of them directly. To try it, download the files, run the script above and ask the simple question from the example.

The edit touches the down_proj matrices of the MLP in layers 15 and 19, 64 columns each. The author published it as an exact diff, and the checkpoint was rebuilt with the author's apply_edit.py, which checks every rebuilt tensor with sha256 against the evaluated checkpoint. Both tensors passed. This is the full multimodal release, so the vision tower is present but unused for text prompts. The card's evidence covers MATH-500 only.

An LLM reads the base model's weights
It finds the repetition loop problem
It edits 131,072 weights in two layers
The edit is rebuilt and hash-checked
The edited model is tested on MATH-500

Good to know:

• Size: 873M parameters, about 1.7 GB in BF16
• Results: the card reports 51.0 percent on MATH-500 over 2,000 generations, against 5.0 percent for the base model on a 25-problem smoke test

Title: MATH-500 accuracy: share of problems solved
Base model smoke test: 5.0%
This model: 51.0%

Title: Cut off at 32k tokens: share of answers that never finished
Base model smoke test: 95%
This model: 31.6%

• License: apache-2.0, and the base model Qwen3.5-0.8B is also Apache 2.0
• Cost to run: none named, no API key or account. The card's tests ran on an NVIDIA H200 GPU
• Data it reaches: it downloads model files from Hugging Face, and the example reads only the prompt you give it
• Not for: tasks outside math, since the card reports results only on MATH-500 and the vision tower goes unused for text prompts

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 873M
• Weights: about 1.7 GB in BF16; about 437 MB at 4-bit (estimate)
• Formats: Safetensors
• Downloads in the last 30 days: 1,177
• Likes: 70
• License: apache-2.0
• Based on: Qwen/Qwen3.5-0.8B
• Quantized versions: 2
• Fine-tunes: 1
• Tags: transformers, safetensors, qwen3_5, image-text-to-text, ml-intern, weight-space-programming, conversational

Links

• Model: https://huggingface.co/ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming
• Files: https://huggingface.co/ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download ML-Intern-lab/Qwen3.5-0.8B-weight-space-programming

Then follow the usage in its model card. Nothing in this kit runs by itself.