ttf-Qwen3.8-27B-GGUF

ttf-Qwen3.8-27B-GGUF

Runs Qwen3.8-27B squeezed to ternary weights as GGUF files that fit on a 12 GB GPU, in an 8.2 GB and a 6.74 GB variant. For people who want a large local model on modest hardware.

Made by oguzeldereli on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This repository holds two GGUF files of Qwen3.8-27B in which every linear layer is reduced to three values, minus one, zero and plus one, with row and column scales. At about 1.6 bits per weight, the model runs entirely on a 12 GB GPU.

The larger file has ternary low-rank adapters on each linear layer. The smaller one skips those adapters and lets the ternary values train during recovery.

Who it suits: It suits people who want a large model at home without a big GPU, and researchers who study extreme low-bit compression of language models.

Industry | Who uses it | How they use it | Example
Local AI | Hobbyists | Chat with a 27B model on a 12 GB card | Ask coding questions offline
Software | Developers | Serve the model as an API on one machine | Point an app at a local port
Research | Quantization researchers | Study ternary conversion and recovery training | Compare scores with full precision
Privacy | Small teams | Keep prompts on their own hardware | Run an assistant without a hosted API

If you want a file that mainline llama.cpp can load directly, you can skip it, since this format needs a patched engine.

Where to use it: You fetch the files on Windows, macOS or Linux with Python, and run them in a patched llama.cpp engine built by the ttf-home tool.

• A terminal chat through the ttf command
• A local server with an OpenAI-compatible API
• A 12 GB GPU, with CUDA when an NVIDIA GPU is found and CPU otherwise

Use case | Who | What they get back
Local chat | Hobbyists | Answers from a 27B model in the terminal
Local API | Developers | An OpenAI-compatible endpoint on your machine
Benchmarking | Researchers | Scores to compare against full precision

How to set it up: You need Python with pip, a GPU with 12 GB, and the ttf-home tool, which builds the patched engine. Mainline llama.cpp does not load these files.

• Python with pip
• A 12 GB GPU, ideally NVIDIA for CUDA
• The ttf-home tool from its GitHub repository

1. Get ttf-home from its GitHub repository and open its folder.
2. Install it with pip.
3. Build the patched engine with the q38 option.
4. Download the GGUF file you want.

pip install -U pip "setuptools>=68"
pip install -e . huggingface_hub
ttf engine build --q38
hf download oguzeldereli/ttf-Qwen3.8-27B-GGUF ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --local-dir models/ttf-27b

How to use it: Once the engine and file are in place, you chat with the model in the terminal or serve it over HTTP on a port you choose.

ttf chat models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf
ttf serve models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --port 8080

Tool | What it does
ttf engine build --q38 | Builds llama.cpp with the TTF patches
ttf chat | Opens a chat with the file you name
ttf serve | Serves the file as an OpenAI-compatible API

Example: You run the chat command with the 8.2 GB file and type a short math word problem. The model replies in the terminal. On the first 200 GSM8K problems the card reports 0.895, the same as full precision.

For newcomers and experts: If you are new, first check that you have a 12 GB GPU, then follow the setup steps in order and try the chat command with one short question.

If you know the field, the conversion folds norm gains into the linears, rotates the residual stream with a randomized Hadamard, and runs ternary GPTQ. Recovery then distills from the full-precision model over 2200 steps and 72M tokens.

Convert layers to ternary weights
Train recovery from full precision
Build the patched llama.cpp engine
Chat or serve the GGUF file

Good to know: The 6.74 GB file needs a newer engine build. If you built the engine earlier, delete its cache folder and rebuild, since an older one stops with an error about the number of tensors.

Benchmark | This model | Full precision
GSM8K, first 200 problems | 0.895 | 0.895
HumanEval | 0.841 | 0.933
IFEval strict, first 200 prompts | 0.735 | 0.787

• License: apache-2.0
• Cost to run: none named, but it needs a 12 GB GPU
• Data it reaches: files you download, and no account or web service is named
• Not for: mainline llama.cpp, which does not load this file type

On Hugging Face, 13 November 2026

• Library: gguf
• Parameters: 33.8B
• Weights: about 16.9 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 320
• Likes: 5
• License: apache-2.0
• Based on: Qwen/Qwen3.8-27B
• Tags: gguf, ternary, 1.58-bit, llama.cpp, conversational

Links

• Model: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF
• Files: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download oguzeldereli/ttf-Qwen3.8-27B-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.