Qwen3.5-0.8B Astra Weight Edit

Qwen3.5-0.8B Astra Weight Edit

Fixes the repeat loops that Qwen3.5-0.8B falls into in thinking mode by editing two weight tensors directly, with no training. A small image-text model for Transformers, vLLM, llama.cpp and the browser.

Made by JonathanColetti on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is Qwen3.5-0.8B with two weight tensors edited directly by a language model, GPT-6 Astra, with no training. The edit targets the repeat loops the base model can fall into when it thinks before answering. The author says the idea comes from researchers at MIT CSAIL and that he only applied their released change and checked it against their checkpoint.

Who it suits: Developers who use the small Qwen3.5-0.8B model with thinking turned on and want to avoid endless repetition, and people curious about editing weights directly.

Industry | Who uses it | How they use it | Example
Software development | Developers | Run a small reasoning model locally | A chat helper on a laptop
Web apps | Front-end developers | Run a small model in the browser | A WebGPU page using the ONNX files
Research | Interpretability researchers | Study weight-space editing | Comparing the edited and base models
Education | Learners | See how a tiny change alters behavior | Reading about the two edited layers

If you run the model with thinking off, you can skip it, because the edit only helps with thinking on.

Where to use it:

• Python with Transformers, vLLM or SGLang on GPU servers
• A browser with WebGPU, or Node.js, through Transformers.js
• Any ONNX Runtime app, and llama.cpp, LM Studio, Ollama or Jan through the GGUF file

Use case | Who | What they get back
Step-by-step reasoning | Developers | Answers without the repeat loops seen in the base model
Browser demos | Web developers | A small model running in WebGPU
Laptop chat | Individuals | A local model through llama.cpp or a desktop app

How to set it up:

• Python and pip for the main route
• About 1.7 GB of disk space in BF16, or about 437 MB at 4-bit (an estimate)
• Thinking turned on, which the chat template switches off unless you ask for it

1. Install the Hugging Face hub tools.
2. Download the model files.
3. Load the model in your runtime with thinking enabled.

pip install -U huggingface_hub
hf download JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit

How to use it: Turn thinking on with enable_thinking set to true, and use the sampling settings the card recommends. The Python example below asks a maths question and prints the answer.

from transformers import AutoModelForImageTextToText, AutoTokenizer
repo = "JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="bfloat16", device_map="auto")
msgs = [{"role": "user", "content": "What is 17 * 23? Please reason step by step, and put your final answer within \\boxed{}."}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=True,
                                 return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=True, temperature=0.6, top_p=0.95, top_k=20)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

To serve it with vLLM, start the server and send chat requests with thinking enabled.

vllm serve JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit
# then send chat requests with: "chat_template_kwargs": {"enable_thinking": true}

Tool | What it does
Transformers, vLLM, SGLang | Run the bf16 safetensors files on Python and GPU servers
Transformers.js | Runs the ONNX files in a browser with WebGPU or in Node.js
llama.cpp, LM Studio, Ollama, Jan | Run the BF16 GGUF file on a laptop or Mac

Example: You ask the model to multiply 17 by 23 and reason step by step, with thinking on. It works through the steps and ends with a boxed answer, without looping on a repeated phrase.

For newcomers and experts: If you are new to this, run the Python example, which needs only Transformers. Remember to turn thinking on, because the edit does nothing with thinking off.

The edit subtracts a loop direction from 64 output columns of the mlp.down_proj weights in layers 15 and 19, about 130 thousand weights or 0.016 percent of the model. Everything else matches the base model byte for byte. The ONNX 4-bit builds agree with bf16 on only about 69 percent of next tokens in a spot check, so treat them as lossy.

Start from Qwen3.5-0.8B
Edit two weight tensors, no training
Turn thinking on in the chat template
Generate without repeat loops

Good to know:

• Thinking: the edit only helps with thinking on
• Credit: the author says all credit goes to the MIT CSAIL researchers, and he is not affiliated with them, OpenAI or Qwen

Title: Language-model decoder size by dtype in the ONNX files, in GB
fp32: 3.0 GB
fp16: 1.5 GB
quantized int8: 0.87 GB
q4: 0.48 GB
q4f16: 0.44 GB

• License: apache-2.0
• Cost to run: no API key or paid service is named, only your own hardware
• Data it reaches: downloading fetches model files, and the model reads the text and images you give it
• Not for: use with thinking off, where the edit gives no benefit

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: transformers
• Parameters: 873M
• Weights: about 1.7 GB in BF16; about 437 MB at 4-bit (estimate)
• Formats: Safetensors, GGUF
• Downloads in the last 30 days: 3,029
• Likes: 27
• License: apache-2.0
• Based on: Qwen/Qwen3.5-0.8B
• Tags: transformers, onnx, safetensors, gguf, qwen3_5, image-text-to-text, weight-editing, weight-space-programming, qwen3.5, math, transformers.js, llama.cpp

Links

• Model: https://huggingface.co/JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit
• Files: https://huggingface.co/JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download JonathanColetti/Qwen3.5-0.8B-Astra-WeightEdit

Then follow the usage in its model card. Nothing in this kit runs by itself.