Qwen3.6-35B-A3B-Escha-W2

Qwen3.6-35B-A3B-Escha-W2

Serves Qwen3.6-35B-A3B locally from a 12.3 GB 2-bit build, through an OpenAI-compatible HTTP API on one 16 or 24 GB NVIDIA GPU. For people who run a large mixture of experts model at home.

Made by EschaLabs on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Escha-W2 squeezes Qwen3.6-35B-A3B down to 2 bits. That base model routes work across 256 experts, and Escha Labs made this build. The weights take 12.3 GB on disk. Per the card, one 24 GB consumer card is enough, and a 16 GB card can do it with fewer parallel streams or a shorter context. A separate runtime repository serves it through an OpenAI-compatible HTTP API.

Who it suits: Developers who want a large model on their own NVIDIA machine, behind an API that existing OpenAI style clients can call.

Industry | Who uses it | How they use it | Example
Software development | Developers who build with local models | Serve the model on a home GPU and point their client at it | Running a coding assistant against a local endpoint
Small teams | Groups that keep data in house | Host one shared model server with several streams | Letting a few teammates query one machine
Research | People who study quantization | Test a 2-bit mixture of experts build | Comparing it with the full precision model

Skip it if you have no NVIDIA GPU of 16 GB or more, or if you are not on Linux x86-64, since the card lists those as requirements. Apple Silicon users are pointed to a separate MLX server.

Where to use it: The weights serve through the Escha runtime on Linux, with an OpenAI compatible interface on port 30000.

• Linux x86-64 with glibc 2.28 or newer, such as Ubuntu 20.04 or later
• An NVIDIA GPU from the Ampere generation to Blackwell, with only the driver installed
• Apple Silicon Macs, through the separate escha-mlx server
• Any client that speaks the OpenAI style API, such as the example opencode config in the repository

Use case | Who | What they get back
Local API server | Developers | An endpoint on port 30000 for chat requests
Tool calling and structured output | Agent builders | Supported by the SGLang engine
Single user decoding | Individual users | Faster long answers with the ZML engine on many cards

How to set it up: Install the Escha runtime first, then download the model and serve it. The card warns that two requirements fail quietly, so read its runtime page too.

• Python 3.12 for the SGLang engine
• A matching PyTorch 2.9 build, since the runtime wheel breaks with a newer one
• About 12.3 GB of disk space

1. Create a Python 3.12 virtual environment and install the Hugging Face tools.
2. Pin torch to a 2.9 release, using the install line on the card.
3. Download the runtime's sglang folder and install its wheel.
4. Download this model and start the server with the runtime's serve script.

python3.12 -m venv .venv && source .venv/bin/activate
pip install -U pip wheel "huggingface_hub[cli]"
hf download EschaLabs/escha-runtime-qwen3moe --include "sglang/*" --local-dir .

You can also fetch this model's files on their own:

pip install -U huggingface_hub
hf download EschaLabs/Qwen3.6-35B-A3B-Escha-W2

How to use it: Once the server runs, send requests to its OpenAI style endpoint. The card advises the SGLang engine unless you have a specific reason to choose ZML.

Engine | Good for | Trade off
sglang | The engine behind the card's numbers, with many streams, tools and structured replies | Needs Python 3.12
zml | A single binary with no Python, and faster single user decoding on many cards | Slower when temperature is above 0, and needs a full 24 GB

Example: You start the SGLang server, then send a chat request to the local endpoint with an OpenAI style client. The model answers over the API, and several requests can run at once.

For newcomers and experts: If you are new to this, follow the runtime page step by step and use the SGLang engine. Check that your GPU and Linux version meet the list first.

If you know the field, the card says the experts are quantized to 2 bits, mixed 2 and 3 bit per projection, with int8 dense layers. ZML decodes faster only at temperature 0, and its first start compiles graphs, which takes from about 75 seconds to minutes depending on the card.

Runtime wheel installs
Model files download
Server starts on port 30000
Client sends OpenAI style requests

Good to know: The card gives its own speed figures. ZML leads by 15 to 26 percent on an RTX 5090 or 3090 and ties on a 5060 Ti. On 16 GB cards, long prompts can fail with an HTTP 500 error under ZML, so use SGLang there.

• License: Apache 2.0
• Cost to run: none named, you supply the NVIDIA GPU
• Data it reaches: the requests you send to the local server
• Not for: macOS or Windows with this runtime, ZML on 16 GB cards with long prompts, or setups without an NVIDIA driver

On Hugging Face, 15 November 2026

• Task: text generation
• Library: mlx
• Parameters: 7.3B
• Weights: about 14.6 GB in I16
• Formats: Safetensors
• Downloads in the last 30 days: 1,425
• Likes: 260
• License: apache-2.0
• Languages: en
• Based on: Qwen/Qwen3.6-35B-A3B
• Quantized versions: 7
• Fine-tunes: 1
• Tags: safetensors, qwen3_5_moe, mixture-of-experts, qwen3, 2-bit, quantization, eschamoe, sglang, apple-silicon, macos, metal, code

Links

• Model: https://huggingface.co/EschaLabs/Qwen3.6-35B-A3B-Escha-W2
• Files: https://huggingface.co/EschaLabs/Qwen3.6-35B-A3B-Escha-W2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download EschaLabs/Qwen3.6-35B-A3B-Escha-W2

Then follow the usage in its model card. Nothing in this kit runs by itself.