ttf-Qwen3.8-Flash-Next-GGUF

ttf-Qwen3.8-Flash-Next-GGUF

Runs Qwen3.8-Flash-Next, a 180B mixture-of-experts model squeezed to ternary weights, from one 40.4 GB GGUF file that is 4.6 times smaller than the FP8 release. For local use with a patched llama.cpp.

Made by oguzeldereli on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is one GGUF file of Qwen3.8-Flash-Next, a mixture-of-experts model with 180B parameters, where every routed expert and dense projection is reduced to ternary values. Next to the FP8 release, the 40.4 GB file takes 4.6 times less space.

The maker converted it layer by layer on one GPU, then ran recovery training with distillation from the FP8 model. The per-layer n-gram embedding table is pruned and product-quantised to 7.96 GB.

Who it suits: It suits people with a large amount of system memory who want an extremely large model locally, and researchers who study ternary compression of language models.

Industry | Who uses it | How they use it | Example
Local AI | Enthusiasts | Chat with a 180B model on a single workstation | Ask coding questions offline
Software | Developers | Test a huge mixture-of-experts model without a cluster | Run a local assistant in the terminal
Research | Compression researchers | Study ternary conversion and recovery training | Compare scores with the FP8 model
Privacy | Small teams | Keep prompts on their own hardware | Run internal drafts locally

If you have less than 64 GB of RAM or want a file that mainline llama.cpp loads, you can skip it.

Where to use it: You download the file on Windows, macOS or Linux with Python, then run it in a patched llama.cpp engine that the ttf-home tool builds.

• A terminal chat through the ttf command
• A single GPU with most experts kept in system memory
• A machine with about 64 GB of RAM

Use case | Who | What they get back
Local chat | Enthusiasts | Replies from a 180B model on your own machine
Code and math | Developers | Answers the card scores on HumanEval and GSM8K
Factual recall tests | Researchers | A look at how ternary weights affect closed-book answers

How to set it up: You need Python with pip, a GPU, about 64 GB of RAM and the ttf-home tool. Mainline llama.cpp does not load this file, because it needs TTF patches for this architecture.

• Python with pip
• A GPU, and about 64 GB of RAM for the experts
• The ttf-home tool from its GitHub repository

1. Get ttf-home from its GitHub repository and open its folder.
2. Install it with pip.
3. Build the patched engine with the q38 option.
4. Download the model files.

pip install -e . huggingface_hub
ttf engine build --q38
hf download oguzeldereli/ttf-Qwen3.8-Flash-Next-GGUF --local-dir models/ttf-flash

How to use it: After the download you open a terminal chat with the GGUF file. Set the thread count to the number of physical cores on your computer.

ttf chat models/ttf-flash/ttf-Qwen3.8-Flash-Next-TQ1_P.gguf

Tool | What it does
ttf engine build --q38 | Builds the patched llama.cpp engine
ttf chat | Opens a chat with the file you name

Example: You start the chat command with the model file and type a short coding question. The model answers in the terminal. The card scores the file at 0.848 on HumanEval.

For newcomers and experts: If you are new, check that you have enough RAM first. Then follow the setup steps in order and send one short question to see how fast it replies.

If you know the field, the conversion applies ternary GPTQ with centred input covariance, adds row and column scales, and attaches a small rank-16 ternary adapter to each linear layer. Then 1000 steps of TTF recovery training use 33M tokens.

Convert each layer to ternary
Train recovery with distillation
Build the patched llama.cpp
Chat with the GGUF file

Good to know: Closed-book factual recall is where ternary weights cost the most, while math and code stay close to FP8 on the card's tests.

Benchmark | This model | FP8 release
GSM8K, first 200 problems | 0.905 | 0.905
HumanEval | 0.848 | 0.927
TriviaQA, 1000 questions | 0.538 | 0.776

The card measured 29.7 tokens per second for decoding on an RTX 3090 Ti with 12 CPU threads, using part of the experts on the GPU.

• License: weights keep the base model's Apache-2.0, and the TTF code is under PolyForm Noncommercial 1.0.0
• Cost to run: none named, but it needs about 64 GB of RAM and a GPU
• Data it reaches: files you download, with no account or web service named
• Not for: mainline llama.cpp, which does not load this file, and the MTP layer and vision tower are not included

On Hugging Face, 13 November 2026

• Library: gguf
• Formats: GGUF
• Downloads in the last 30 days: 190
• Likes: 5
• License: apache-2.0
• Based on: Qwen/Qwen3.8-Flash-Next
• Tags: gguf, ternary, 1.58-bit, llama.cpp

Links

• Model: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-Flash-Next-GGUF
• Files: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-Flash-Next-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download oguzeldereli/ttf-Qwen3.8-Flash-Next-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.