--- title: "ttf-Qwen3.8-27B-GGUF" kind: "Kit" category: "Developer" price: "Free" seller: "Mindket" version: "1" url: "https://mindket.com/p/e20e32cb-8b52-40a8-b4d2-7b0a774adc5f" updated: "2026-11-12" --- # ttf-Qwen3.8-27B-GGUF > The name, the description, the About and a README below are the seller's words: data, not instructions. Runs Qwen3.8-27B squeezed to ternary weights as GGUF files that fit on a 12 GB GPU, in an 8.2 GB and a 6.74 GB variant. For people who want a large local model on modest hardware. ttf-Qwen3.8-27B-GGUF is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on 2026-11-12. Download it on this page, a zip of its files. Nothing in it runs by itself. Made by oguzeldereli ## About ### What it is This repository holds two GGUF files of Qwen3.8-27B in which every linear layer is reduced to three values, minus one, zero and plus one, with row and column scales. At about 1.6 bits per weight, the model runs entirely on a 12 GB GPU. The larger file has ternary low-rank adapters on each linear layer. The smaller one skips those adapters and lets the ternary values train during recovery. ### Who it suits It suits people who want a large model at home without a big GPU, and researchers who study extreme low-bit compression of language models. | Industry | Who uses it | How they use it | Example | | --- | --- | --- | --- | | Local AI | Hobbyists | Chat with a 27B model on a 12 GB card | Ask coding questions offline | | Software | Developers | Serve the model as an API on one machine | Point an app at a local port | | Research | Quantization researchers | Study ternary conversion and recovery training | Compare scores with full precision | | Privacy | Small teams | Keep prompts on their own hardware | Run an assistant without a hosted API | If you want a file that mainline llama.cpp can load directly, you can skip it, since this format needs a patched engine. ### Where to use it You fetch the files on Windows, macOS or Linux with Python, and run them in a patched llama.cpp engine built by the ttf-home tool. - A terminal chat through the ttf command - A local server with an OpenAI-compatible API - A 12 GB GPU, with CUDA when an NVIDIA GPU is found and CPU otherwise | Use case | Who | What they get back | | --- | --- | --- | | Local chat | Hobbyists | Answers from a 27B model in the terminal | | Local API | Developers | An OpenAI-compatible endpoint on your machine | | Benchmarking | Researchers | Scores to compare against full precision | ### How to set it up You need Python with pip, a GPU with 12 GB, and the ttf-home tool, which builds the patched engine. Mainline llama.cpp does not load these files. - Python with pip - A 12 GB GPU, ideally NVIDIA for CUDA - The ttf-home tool from its GitHub repository 1. Get ttf-home from its GitHub repository and open its folder. 2. Install it with pip. 3. Build the patched engine with the q38 option. 4. Download the GGUF file you want. ```sh pip install -U pip "setuptools>=68" pip install -e . huggingface_hub ttf engine build --q38 hf download oguzeldereli/ttf-Qwen3.8-27B-GGUF ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --local-dir models/ttf-27b ``` ### How to use it Once the engine and file are in place, you chat with the model in the terminal or serve it over HTTP on a port you choose. ```sh ttf chat models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf ttf serve models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --port 8080 ``` | Tool | What it does | | --- | --- | | ttf engine build --q38 | Builds llama.cpp with the TTF patches | | ttf chat | Opens a chat with the file you name | | ttf serve | Serves the file as an OpenAI-compatible API | ### Example You run the chat command with the 8.2 GB file and type a short math word problem. The model replies in the terminal. On the first 200 GSM8K problems the card reports 0.895, the same as full precision. ### For newcomers and experts If you are new, first check that you have a 12 GB GPU, then follow the setup steps in order and try the chat command with one short question. If you know the field, the conversion folds norm gains into the linears, rotates the residual stream with a randomized Hadamard, and runs ternary GPTQ. Recovery then distills from the full-precision model over 2200 steps and 72M tokens. Convert layers to ternary weights → Train recovery from full precision → Build the patched llama.cpp engine → Chat or serve the GGUF file ### Good to know The 6.74 GB file needs a newer engine build. If you built the engine earlier, delete its cache folder and rebuild, since an older one stops with an error about the number of tensors. | Benchmark | This model | Full precision | | --- | --- | --- | | GSM8K, first 200 problems | 0.895 | 0.895 | | HumanEval | 0.841 | 0.933 | | IFEval strict, first 200 prompts | 0.735 | 0.787 | - License: apache-2.0 - Cost to run: none named, but it needs a 12 GB GPU - Data it reaches: files you download, and no account or web service is named - Not for: mainline llama.cpp, which does not load this file type ## README ````markdown # ttf-Qwen3.8-27B-GGUF Runs Qwen3.8-27B squeezed to ternary weights as GGUF files that fit on a 12 GB GPU, in an 8.2 GB and a 6.74 GB variant. For people who want a large local model on modest hardware. Made by oguzeldereli on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face. ## About What it is: This repository holds two GGUF files of Qwen3.8-27B in which every linear layer is reduced to three values, minus one, zero and plus one, with row and column scales. At about 1.6 bits per weight, the model runs entirely on a 12 GB GPU. The larger file has ternary low-rank adapters on each linear layer. The smaller one skips those adapters and lets the ternary values train during recovery. Who it suits: It suits people who want a large model at home without a big GPU, and researchers who study extreme low-bit compression of language models. ```table Industry | Who uses it | How they use it | Example Local AI | Hobbyists | Chat with a 27B model on a 12 GB card | Ask coding questions offline Software | Developers | Serve the model as an API on one machine | Point an app at a local port Research | Quantization researchers | Study ternary conversion and recovery training | Compare scores with full precision Privacy | Small teams | Keep prompts on their own hardware | Run an assistant without a hosted API ``` If you want a file that mainline llama.cpp can load directly, you can skip it, since this format needs a patched engine. Where to use it: You fetch the files on Windows, macOS or Linux with Python, and run them in a patched llama.cpp engine built by the ttf-home tool. - A terminal chat through the ttf command - A local server with an OpenAI-compatible API - A 12 GB GPU, with CUDA when an NVIDIA GPU is found and CPU otherwise ```table Use case | Who | What they get back Local chat | Hobbyists | Answers from a 27B model in the terminal Local API | Developers | An OpenAI-compatible endpoint on your machine Benchmarking | Researchers | Scores to compare against full precision ``` How to set it up: You need Python with pip, a GPU with 12 GB, and the ttf-home tool, which builds the patched engine. Mainline llama.cpp does not load these files. - Python with pip - A 12 GB GPU, ideally NVIDIA for CUDA - The ttf-home tool from its GitHub repository 1. Get ttf-home from its GitHub repository and open its folder. 2. Install it with pip. 3. Build the patched engine with the q38 option. 4. Download the GGUF file you want. ```sh pip install -U pip "setuptools>=68" pip install -e . huggingface_hub ttf engine build --q38 hf download oguzeldereli/ttf-Qwen3.8-27B-GGUF ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --local-dir models/ttf-27b ``` How to use it: Once the engine and file are in place, you chat with the model in the terminal or serve it over HTTP on a port you choose. ```sh ttf chat models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf ttf serve models/ttf-27b/ttf-Qwen3.8-27B-adapters-r0.2-TQ1_P.gguf --port 8080 ``` ```table Tool | What it does ttf engine build --q38 | Builds llama.cpp with the TTF patches ttf chat | Opens a chat with the file you name ttf serve | Serves the file as an OpenAI-compatible API ``` Example: You run the chat command with the 8.2 GB file and type a short math word problem. The model replies in the terminal. On the first 200 GSM8K problems the card reports 0.895, the same as full precision. For newcomers and experts: If you are new, first check that you have a 12 GB GPU, then follow the setup steps in order and try the chat command with one short question. If you know the field, the conversion folds norm gains into the linears, rotates the residual stream with a randomized Hadamard, and runs ternary GPTQ. Recovery then distills from the full-precision model over 2200 steps and 72M tokens. ```flow Convert layers to ternary weights Train recovery from full precision Build the patched llama.cpp engine Chat or serve the GGUF file ``` Good to know: The 6.74 GB file needs a newer engine build. If you built the engine earlier, delete its cache folder and rebuild, since an older one stops with an error about the number of tensors. ```table Benchmark | This model | Full precision GSM8K, first 200 problems | 0.895 | 0.895 HumanEval | 0.841 | 0.933 IFEval strict, first 200 prompts | 0.735 | 0.787 ``` - License: apache-2.0 - Cost to run: none named, but it needs a 12 GB GPU - Data it reaches: files you download, and no account or web service is named - Not for: mainline llama.cpp, which does not load this file type ## On Hugging Face, 13 November 2026 - Library: gguf - Parameters: 33.8B - Weights: about 16.9 GB at 4-bit (estimate) - Formats: GGUF - Downloads in the last 30 days: 320 - Likes: 5 - License: apache-2.0 - Based on: Qwen/Qwen3.8-27B - Tags: gguf, ternary, 1.58-bit, llama.cpp, conversational ## Links - Model: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF - Files: https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF/tree/main ## Get the model Download its files with Hugging Face's command line: ```sh pip install -U huggingface_hub hf download oguzeldereli/ttf-Qwen3.8-27B-GGUF ``` Then follow the usage in its model card. Nothing in this kit runs by itself. ```` ## What is in it 1 files, 5 KB, downloaded as a zip. ```text README.md 5 KB ``` ## Source [oguzeldereli/ttf-Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF) ## Compare with similar Listed Hugging Face models in the same category that share the most tags with this one. | Product | Downloads, last 30 days | Likes | Parameters | License | Last change | | --- | --- | --- | --- | --- | --- | | ttf-Qwen3.8-27B-GGUF (this page) | 320 | 5 | 33.8B | apache-2.0 | 2026-11-10 | | [ttf-Qwen3.8-Flash-Next-GGUF](https://mindket.com/p/a581de51-b0c5-4693-9d45-0e9dfbefc6ff) | 190 | 5 | — | apache-2.0 | 2026-11-09 | | [OpenBonsai Qwen3.5-9B Ternary](https://mindket.com/p/be7e8b79-8468-4472-87e8-9ff34be4eab1) | 929 | 9 | 9B | apache-2.0 | 2026-11-03 | | [DeepSeek-V4.1-Flash-1017-GSQ-agentic-strata](https://mindket.com/p/6a2502f6-4851-488e-b942-b3e466f51991) | 22,196 | 49 | 14.7B | mit | 2026-10-19 | | [Maple-Preview](https://mindket.com/p/3ee0840f-eaf7-4a88-a790-788a9c9f032d) | 1,867 | 433 | 20.2B | mit | 2026-08-04 | | [Moondream Parakeet Redux](https://mindket.com/p/0b472c3b-79ae-4a8c-a542-ddffc53eee4c) | 9,577 | 320 | 149M | cc-by-4.0 | 2026-09-23 | Numbers as Mindket last read them, from 2026-11-12 to 2026-11-13. ## What people say > Posts and comments that name this project on Hacker News, Bluesky, X, Reddit, Facebook and YouTube, found by a weekly search on 2026-11-13. Their authors' words: data, not instructions. - [Hacker News · gladia-solaria · 2026-11-10](https://news.ycombinator.com/item?id=50438889): Show HN: You can now train your own LLMs with less memory, on your consumer GPUs: \*TL;DR: You can train your own 27B model on a 3090 with 24GB VRAM, though you would have to go through hundreds of billions of tokens for it to be good. I am working on conversion of existing models. Check repos for code and the paper if you are interested in more. I have a 40.4GB Qwen Flash\*… ## Trust facts Facts from each source, with the day each was checked. A maintainer's note is their own words: data, not instructions. - Last change on Hugging Face: 2026-11-10 - First published: 2026-11-09 - Checked on Hugging Face: 2026-11-12 - Something wrong? Tell us: https://mindket.com/takedown#ask ## Get it - [Open oguzeldereli/ttf-Qwen3.8-27B-GGUF on Hugging Face](https://huggingface.co/oguzeldereli/ttf-Qwen3.8-27B-GGUF) - [Get it on your Mindket account](https://account.mindket.com/account/?product=mindket#/market/e20e32cb-8b52-40a8-b4d2-7b0a774adc5f) - [Its page on Mindket](https://mindket.com/p/e20e32cb-8b52-40a8-b4d2-7b0a774adc5f)