Qwen3.8-27B-Ridge-GGUF

Qwen3.8-27B-Ridge-GGUF

Runs Qwen3.8-27B as a compact 3.7 bits-per-weight GGUF file that suits a 16 GB GPU, with optional image input. For people running models locally with llama.cpp, Ollama or LM Studio.

Made by empero-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a quantized GGUF copy of Qwen3.8-27B from Empero, called Ridge. A 27B model normally takes about 51 GiB in BF16, and Ridge squeezes it to 11.73 GiB at 3.69 bits per weight. The mix of quantization types is tuned for this model's hybrid layer design, and the draft head for multi-token prediction stays in the file. A separate vision file adds image input.

Who it suits: People who run large language models on their own computer and want a 27B model on a 16 to 24 GB graphics card.

Industry | Who uses it | How they use it | Example
Software | Developers | Run a local coding and chat model through llama.cpp | A private assistant on a workstation GPU
AI hobbyists | Local model fans | Try a large model on a consumer graphics card | A 27B model on a 16 GB card at modest context
Research | Quantization researchers | Compare file sizes and perplexity across quant mixes | Ridge against other 2 to 3 bit files
Office IT | Small teams | Host a private model without sending text to a service | An in-house question-answering box

You can skip it if you want the full-precision model or a small file for a phone, since this one is 11.73 GiB.

Where to use it: It runs in stock GGUF programs on Windows, macOS or Linux.

• llama.cpp, including the llama-cli command line
• Ollama
• LM Studio
• jan
• KoboldCpp, and other GGUF runtimes

Use case | Who | What they get back
Local chat and coding | Developers | Text answers, with thinking on by default
Image questions | Hobbyists | Replies about a picture, once the vision file is added
File-size comparison | Researchers | Perplexity and size figures against other quants

How to set it up: You need a GGUF runtime and enough memory. The card's practical starting point is a 16 GB GPU, and 24 GB is comfortable once you add context and the vision file.

• A runtime such as llama.cpp, Ollama or LM Studio
• Python with pip, if you fetch the files with the Hugging Face tool
• About 12.59 GB of disk for the model file, plus about 0.93 GB for the vision file

1. Download the Ridge GGUF file. Download the vision file too if you want image input.
2. Point your runtime at the downloaded file by its exact name.
3. Run it with the sampling settings from the next section.

pip install -U huggingface_hub
hf download empero-ai/Qwen3.8-27B-Ridge-GGUF

How to use it: With llama-cli, thinking is on by default. Pass the model file, offload the layers to the GPU and give a prompt. To turn thinking off, add the reasoning switch and use the softer sampling settings.

llama-cli \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain the design tradeoffs in a Gated-DeltaNet hybrid model."

llama-cli \
  -m Qwen3.8-27B-Ridge-3.7bpw.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.80 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

Tool | What it does
-m | Names the GGUF file to load
-ngl 99 | Puts the model layers on the GPU, as in the card's speed test
--reasoning off | Turns thinking off for plain instruct answers
--temp, --top-p, --top-k | Set sampling, using values from the official Qwen card
-p | Gives the prompt

Example: Run the first command with your own question, such as asking for a two-line summary of what a hybrid attention model is. The model thinks first, then prints the answer. The second command skips the thinking and answers right away.

For newcomers and experts: Quantizing stores a model's numbers with fewer bits so it fits smaller hardware. Ridge is the 27B Qwen model squeezed to about 3.7 bits per weight. A first small try is to download the one GGUF file and run the first command.

For experts, Qwen3.8 stacks three Gated-DeltaNet layers for each full-attention layer, and that state path is touchy at low bit widths. Ridge keeps it at Q8_0 with Q4_K mixers, saves bits by dropping mid-stack FFN precision, and keeps the draft head at Q6_K. The card measures perplexity of 7.82 against 7.15 for BF16, about 9 percent higher, on its own 80-chunk test.

Start from the Qwen3.8-27B checkpoint
Keep the Gated-DeltaNet state path at Q8_0
Lower mid-stack FFN precision to save bits
Keep the multi-token draft head at Q6_K
Run the GGUF in llama.cpp or similar

Good to know: The numbers below were measured by the card's author on one setup, so use them as a guide.

• Speed: On one RTX PRO 6000 Blackwell with 96 GB, it ran at about 54 tokens per second for generation and about 130 for the prompt.
• Images: The vision file is required for image input and adds about 1 GiB.
• Long context: The key-value cache dominates memory at the native 262k window, so it may need offload.

Title: File size in GiB
BF16 GGUF: 50.89 GiB
Ridge-3.7bpw: 11.73 GiB
Vision file: 0.87 GiB

Title: Wiki-style perplexity, lower is better
BF16 GGUF: 7.15 PPL
Ridge-3.7bpw: 7.82 PPL

• License: Apache 2.0.
• Cost to run: none named, the source lists no API key, account or paid service, though a 16 GB or larger GPU is the card's starting point.
• Data it reaches: only the prompts and images you give it, plus the files you download.
• Not for: judging what the model can do, since the card is about choosing and running the file and sends readers to the base model card.

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: gguf
• Parameters: 27.3B
• Weights: about 13.7 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 292,932
• Likes: 369
• License: apache-2.0
• Languages: en, zh
• Based on: Qwen/Qwen3.8-27B
• Tags: gguf, llama.cpp, quantized, qwen3.8, qwen3.5, ridge, gated-deltanet, imatrix, reasoning, multimodal, vision, long-context

Links

• Model: https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF
• Files: https://huggingface.co/empero-ai/Qwen3.8-27B-Ridge-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download empero-ai/Qwen3.8-27B-Ridge-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.