Mindket

Qwen3.8-27B-ASCII-Condensed

Newversion 1 · updated

Overview

Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code with llama.cpp.

Qwen3.8-27B-ASCII-Condensed is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.

At a glance

You need
  • A llama.cpp build that includes llama-cli
  • A 16 GB graphics card, since the context limits were measured on one
  • Python, to fetch the files with the Hugging Face tools
Runs in
  • llama.cpp tools such as llama-cli, on Windows, macOS or Linux
  • A graphics card with 16 GB, where the author tested an RTX 5070 Ti
  • A computer with enough system RAM to hold the embedding table
  • Cards with more VRAM, where the context limits move upward
Start with
pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed

Platform

WindowsOperating system
macOSOperating system
LinuxOperating system

About

What it is

Qwen3.8-27B-ASCII-Condensed is a GGUF build of Qwen3.8-27B whose vocabulary keeps only ASCII tokens, so more of a 16 GB graphics card is left for long context. On an RTX 5070 Ti the author reached a context of 135,168 tokens, while the same quantization without the cut manages 114,688. No weights were retrained or requantized.

Who it suits

It suits people who run large local models with llama.cpp on one 16 GB graphics card and need a long prompt window for English text or code. Anyone who works mostly in other languages should look elsewhere.

IndustryWho uses itHow they use itExample
Software developmentDevelopers running a local coding assistantFeed long files into the promptUp to 135,168 tokens fit on a 16 GB card in the author's tests
Research and writingAnalysts who work in EnglishQuestion long documents on their own machineA long document stays in context on a single card
Local AI hobbyEnthusiasts with one consumer GPURun a 27B model at long context without a larger cardAn RTX 5070 Ti reached 135,168 tokens

Where to use it

  • llama.cpp tools such as llama-cli, on Windows, macOS or Linux
  • A graphics card with 16 GB, where the author tested an RTX 5070 Ti
  • A computer with enough system RAM to hold the embedding table
  • Cards with more VRAM, where the context limits move upward
Use caseWhoWhat they get back
Long-prompt chatLocal model usersA usable context of up to 135,168 tokens on a 16 GB card
Faster decoding with MTPUsers who accept a shorter contextAbout 1.8x faster decoding at up to 77,824 tokens
Code and English text workDevelopersThe same tokenizer output as the base model on ASCII text

How to set it up

You need llama.cpp and a graphics card with 16 GB of memory.

  • A llama.cpp build that includes llama-cli
  • A 16 GB graphics card, since the context limits were measured on one
  • Python, to fetch the files with the Hugging Face tools
  1. Install the Hugging Face tools with Python.
  2. Download the model files with the second command below.
  3. Check that your llama.cpp build is stock, or patched if you want mixed cache types.
  4. Run llama-cli with the quickstart flags.
pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed

The quickstart command from the card:

llama-cli -m Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf \
  -ngl 99 -fa on -ctk q4_0 -ctv q4_0 \
  -ot "token_embd.weight=CPU" \
  -c 135168 -ub 128 -b 512

How to use it

Run the quickstart command, then chat or prompt as with any llama.cpp model. The embedding table stays in system RAM, which costs about 1% of decode speed and frees its whole footprint on the card.

ToolWhat it does
-ot token_embd.weight=CPUKeeps the embedding table in system RAM
-ctk and -ctvSet the cache types, which must match on stock builds
-cSets the context size, 135168 in the quickstart
--spec-type draft-mtpTurns on MTP speculative decoding, about 1.8x faster at a smaller context

For MTP, add these flags:

--spec-type draft-mtp --spec-draft-n-max 3 -ctkd q4_0 -ctvd q4_0

Example

You run the quickstart command on a 16 GB card and send a prompt about 15K tokens deep. The author measured decoding near 47.3 tokens per second there, and near 33.1 at 90K, because decoding slows as the cache fills.

For newcomers and experts

Think of it as the same 27B model with the unused part of its dictionary removed, which leaves room for a longer conversation. The first thing to try is the quickstart command with a short prompt, then a longer one.

Underneath, non-ASCII rows were removed from the token embedding and output tensors, and the tokenizer was rewritten to match. The vocabulary shrank from 248,320 to 129,006 rows, and 864 of 866 tensors match the base quant bit for bit. Embedding is a gather and the output head runs over the whole vocabulary for every token, so shrinking both frees VRAM for the KV cache. The trade-off is that non-ASCII text costs extra tokens.

  1. Start from the UD-IQ4_XS quantization
  2. Drop non-ASCII rows from two tensors
  3. Rewrite the tokenizer to match
  4. Freed VRAM becomes KV cache
  5. Longer context fits on a 16 GB card

Good to know

Context ceilings depend on the cache types you choose, and they were measured on a 16 GB card with a real prompt of about 15K tokens.

Max context by cache type
q4_0 + q4_0135168 tokens
q5_1 + q4_0118784 tokens
q8_0 + q4_098304 tokens
q8_0 + q8_077824 tokens
Speed on an RTX 5070 Ti at q4_0 and q4_0
Prompt depthPrefillDecode
15K1,501 t/s47.3 t/s
90K930 t/s33.1 t/s
Token cost of non-ASCII text
TextCost
English, code, ASCII punctuationUnchanged
Unaccented Latin scriptUnchanged
Accented Latin lettersAbout 2 tokens per accented character
CJK, Cyrillic, Arabic, ThaiAbout 3 tokens per character
  • Text only: no mmproj file ships with it, and vision has not been tested
  • Mixed cache types: they need a patched llama.cpp, and on stock builds a mismatch silently disables flash attention and slows prefill about 37 times
  • MTP draft model: it keeps its own cache and defaults to f16, so pass -ctkd and -ctvd yourself
  • Not retrained: it prunes the vocabulary of an existing quantization, with no distillation or new quantization step
  • License: Apache 2.0, passed down from the base model, and a LICENSE file comes with it
  • Cost to run: none named, but a 16 GB graphics card was used for the limits
  • Data it reaches: none named, since it is a local model file
  • Not for: work mostly in non-English languages, or image input

README and files

README

Qwen3.8-27B-ASCII-Condensed

Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code with llama.cpp.

Made by bsaleh03 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Qwen3.8-27B-ASCII-Condensed is a GGUF build of Qwen3.8-27B whose vocabulary keeps only ASCII tokens, so more of a 16 GB graphics card is left for long context. On an RTX 5070 Ti the author reached a context of 135,168 tokens, while the same quantization without the cut manages 114,688. No weights were retrained or requantized.

Who it suits: It suits people who run large local models with llama.cpp on one 16 GB graphics card and need a long prompt window for English text or code. Anyone who works mostly in other languages should look elsewhere.

Industry | Who uses it | How they use it | Example
Software development | Developers running a local coding assistant | Feed long files into the prompt | Up to 135,168 tokens fit on a 16 GB card in the author's tests
Research and writing | Analysts who work in English | Question long documents on their own machine | A long document stays in context on a single card
Local AI hobby | Enthusiasts with one consumer GPU | Run a 27B model at long context without a larger card | An RTX 5070 Ti reached 135,168 tokens

Where to use it:

  • llama.cpp tools such as llama-cli, on Windows, macOS or Linux
  • A graphics card with 16 GB, where the author tested an RTX 5070 Ti
  • A computer with enough system RAM to hold the embedding table
  • Cards with more VRAM, where the context limits move upward
Use case | Who | What they get back
Long-prompt chat | Local model users | A usable context of up to 135,168 tokens on a 16 GB card
Faster decoding with MTP | Users who accept a shorter context | About 1.8x faster decoding at up to 77,824 tokens
Code and English text work | Developers | The same tokenizer output as the base model on ASCII text

How to set it up: You need llama.cpp and a graphics card with 16 GB of memory.

  • A llama.cpp build that includes llama-cli
  • A 16 GB graphics card, since the context limits were measured on one
  • Python, to fetch the files with the Hugging Face tools
  1. Install the Hugging Face tools with Python.
  2. Download the model files with the second command below.
  3. Check that your llama.cpp build is stock, or patched if you want mixed cache types.
  4. Run llama-cli with the quickstart flags.
pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed

The quickstart command from the card:

llama-cli -m Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf \
  -ngl 99 -fa on -ctk q4_0 -ctv q4_0 \
  -ot "token_embd.weight=CPU" \
  -c 135168 -ub 128 -b 512

How to use it: Run the quickstart command, then chat or prompt as with any llama.cpp model. The embedding table stays in system RAM, which costs about 1% of decode speed and frees its whole footprint on the card.

Tool | What it does
-ot token_embd.weight=CPU | Keeps the embedding table in system RAM
-ctk and -ctv | Set the cache types, which must match on stock builds
-c | Sets the context size, 135168 in the quickstart
--spec-type draft-mtp | Turns on MTP speculative decoding, about 1.8x faster at a smaller context

For MTP, add these flags:

--spec-type draft-mtp --spec-draft-n-max 3 -ctkd q4_0 -ctvd q4_0

Example: You run the quickstart command on a 16 GB card and send a prompt about 15K tokens deep. The author measured decoding near 47.3 tokens per second there, and near 33.1 at 90K, because decoding slows as the cache fills.

For newcomers and experts: Think of it as the same 27B model with the unused part of its dictionary removed, which leaves room for a longer conversation. The first thing to try is the quickstart command with a short prompt, then a longer one.

Underneath, non-ASCII rows were removed from the token embedding and output tensors, and the tokenizer was rewritten to match. The vocabulary shrank from 248,320 to 129,006 rows, and 864 of 866 tensors match the base quant bit for bit. Embedding is a gather and the output head runs over the whole vocabulary for every token, so shrinking both frees VRAM for the KV cache. The trade-off is that non-ASCII text costs extra tokens.

Start from the UD-IQ4_XS quantization
Drop non-ASCII rows from two tensors
Rewrite the tokenizer to match
Freed VRAM becomes KV cache
Longer context fits on a 16 GB card

Good to know: Context ceilings depend on the cache types you choose, and they were measured on a 16 GB card with a real prompt of about 15K tokens.

Title: Max context by cache type
q4_0 + q4_0: 135168 tokens
q5_1 + q4_0: 118784 tokens
q8_0 + q4_0: 98304 tokens
q8_0 + q8_0: 77824 tokens
Title: Speed on an RTX 5070 Ti at q4_0 and q4_0
Prompt depth | Prefill | Decode
15K | 1,501 t/s | 47.3 t/s
90K | 930 t/s | 33.1 t/s
Title: Token cost of non-ASCII text
Text | Cost
English, code, ASCII punctuation | Unchanged
Unaccented Latin script | Unchanged
Accented Latin letters | About 2 tokens per accented character
CJK, Cyrillic, Arabic, Thai | About 3 tokens per character
  • Text only: no mmproj file ships with it, and vision has not been tested
  • Mixed cache types: they need a patched llama.cpp, and on stock builds a mismatch silently disables flash attention and slows prefill about 37 times
  • MTP draft model: it keeps its own cache and defaults to f16, so pass -ctkd and -ctvd yourself
  • Not retrained: it prunes the vocabulary of an existing quantization, with no distillation or new quantization step
  • License: Apache 2.0, passed down from the base model, and a LICENSE file comes with it
  • Cost to run: none named, but a 16 GB graphics card was used for the limits
  • Data it reaches: none named, since it is a local model file
  • Not for: work mostly in non-English languages, or image input

On Hugging Face, 16 November 2026

  • Task: text generation
  • Parameters: 26.1B
  • Weights: about 13 GB at 4-bit (estimate)
  • Formats: GGUF
  • Downloads in the last 30 days: 1,677
  • Likes: 35
  • License: apache-2.0
  • Based on: Qwen/Qwen3.8-27B
  • Tags: gguf, qwen3, long-context, vocabulary-pruning, llama.cpp, text-generation, imatrix, conversational

Links

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed

Then follow the usage in its model card. Nothing in this kit runs by itself.

Files
README.md  7 KB

1 file, 7 KB, downloaded as a zip.

What’s new

  • Version 1 · 2026-11-16

Compare with similar

Listed Hugging Face models in the same category that share the most tags with this one.

ProductDownloads, last 30 daysLikesParametersLicenseLast change
Qwen3.8-27B-ASCII-Condensed This page1,6773526.1Bapache-2.0
Sharp-Spark-X2.5-4B-GGUFShares long-context · imatrix · llama.cpp875,3461644.1Bapache-2.0
Sharp-Spark-X2.6-4B-GGUFShares long-context · imatrix · llama.cpp5,044234.1Bapache-2.0
Qwen3.8-27B-Ridge-GGUFShares long-context · imatrix · llama.cpp292,93236927.3Bapache-2.0
MiniCPM5-2B-GGUFShares long-context · llama.cpp · gguf481,1193932.5Bapache-2.0
Qwen3-30B-A3B-GGUF by unslothShares qwen3 · imatrix · gguf53,66429430.5Bapache-2.0
Numbers as Mindket last read them, from 2026-11-12 to 2026-11-16.

Reviews

No ratings yet

Write a review

No reviews yet.

More from Mindket

See all from Mindket

all-mpnet-base-v2

sentence-transformers · Kit

Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…

New

multilingual-e5-base

intfloat · Kit

Turns text in about 100 languages into vectors so you can compare meaning across them, for search and matching. Made for developers who build multilingual retr…

New

sprite-gen

aldegad · Skill

Turns one base drawing of a character into game-ready sprites, either an atlas with a manifest or transparent motion loops, for game developers working in Code…

New4097

Similar in Developer

See all in Developer

VibeSkills

foryourhealth111-pixel · Skill

Picks the right skills from a large installed pool and breaks each task into checkable steps for your coding agent. In its own benchmark, mean reward rose 21.1…

New3821

claude-bughunter

elementalsouls · Skill

Gives Claude Code 83 skills and 15 slash commands for bug bounty hunting and authorized red-team work, from web vulnerability classes to enterprise identity at…

New6853

Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their…

New