Qwen3.6-35B-A3B-coding-reap50-GGUF

Qwen3.6-35B-A3B-coding-reap50-GGUF

Writes code with Qwen3.6-35B-A3B cut to half its experts and packed as GGUF files for llama.cpp. Made for developers who want a coding model that fits in less memory than the full model.

Made by anik-jha on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This model takes Qwen3.6-35B-A3B and drops half the experts in every layer, keeping 128 of the original 256. REAP scoring on code-focused data decided which ones stay. The pruned model was then quantized with an importance matrix into GGUF files for llama.cpp. The result has about 19B parameters, and a paper titled Half the Experts, All the Code goes with it.

Who it suits: It suits developers who want a local coding model that fits where the full Qwen3.6-35B-A3B does not.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local code assistant with less memory | Ask for a function and review the code
AI research | Pruning researchers | Study expert pruning for mixture-of-experts models | Compare pruned and full-model coding scores
Education | Programming students | Practice with a coding model offline | Get help with a small coding exercise
Tooling | Tool builders | Back a code editor with a local llama.cpp server | Complete code at the same memory budget

If you want a general assistant for non-coding work, you can skip it, since the card warns against that use.

Where to use it: It runs anywhere llama.cpp runs, on Windows, macOS or Linux.

• llama.cpp and apps built on it
• Local coding workflows on machines with enough memory for the file size
• Windows, macOS or Linux

Use case | Who | What they get back
Code generation | Developers | Code written by the pruned model
Benchmark comparison | Researchers | Scores to compare with the full model's low-bit versions
Reproducing the pruning | Model builders | An expert selection file in the moep repository

How to set it up: You need Python for the download tool and a llama.cpp build. Pick one of the two GGUF files, which are 11.4 GB and 13.3 GB.

1. Install the Hugging Face hub tools.
2. Download the repository files.
3. Point llama.cpp at the GGUF file you chose.

pip install -U huggingface_hub
hf download anik-jha/Qwen3.6-35B-A3B-coding-reap50-GGUF

How to use it: Load the GGUF file in llama.cpp, or an app built on it, and give it coding tasks. The card's tests turned reasoning off.

The card part shown has no llama.cpp command, so use your own llama.cpp setup for the exact call.

File | Size | Note
Q4_K_M with importance matrix | 11.4 GB | Smaller file, slightly lower scores
Q5_K_M with importance matrix | 13.3 GB | Larger file, the best scores in the card's table

Example: You load the Q4_K_M file in llama.cpp and ask it to write a small Python function. It returns code. The card does not show a sample reply, only benchmark scores for the file.

For newcomers and experts: A mixture-of-experts model has many specialist blocks, and only a few run for each token. Pruning removes half of them. To try it, start with the smaller file and a simple coding request.

The pruning used REAP scoring on code-heavy data, so general knowledge suffers while coding holds up. The card reports that general perplexity roughly doubles. Scores come from greedy pass@1 with EvalPlus against llama.cpp, with reasoning off.

Start with Qwen3.6-35B-A3B
Score experts on coding data with REAP
Remove half of the experts
Quantize with an importance matrix
Run the GGUF file in llama.cpp

Good to know: At a similar memory budget, the card says these files beat a 2-bit version of the full model on HumanEval+.

Title: HumanEval+ score, greedy pass@1
Q5_K_M pruned: 0.915 score
Q4_K_M pruned: 0.902 score
Full model IQ2_M: 0.896 score
Full model Q8: 0.890 score

• Trade-off: general perplexity roughly doubles, so treat it as a coder and keep it out of general assistant roles.
• The full-model rows in the card are for comparison only and are not included here.
• Reproduction: the expert selection file is in the moep repository, so the pruning can be redone from the base model.
• License: Apache 2.0, as a derivative of Qwen3.6-35B-A3B.
• Cost to run: none named.
• Data it reaches: the prompts and code you give it, handled on your own machine.
• Not for: use as a general assistant.

On Hugging Face, 13 November 2026

• Library: llama.cpp
• Parameters: 18.5B
• Weights: about 9.3 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 3,898
• Likes: 16
• License: apache-2.0
• Based on: Qwen/Qwen3.6-35B-A3B
• Tags: llama.cpp, gguf, mixture-of-experts, pruning, code, imatrix, conversational

Links

• Model: https://huggingface.co/anik-jha/Qwen3.6-35B-A3B-coding-reap50-GGUF
• Files: https://huggingface.co/anik-jha/Qwen3.6-35B-A3B-coding-reap50-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download anik-jha/Qwen3.6-35B-A3B-coding-reap50-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.