all-mpnet-base-v2
sentence-transformers · Kit
Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…
Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code with llama.cpp.
Qwen3.8-27B-ASCII-Condensed is a kit in the Developer category, listed by Mindket. It is free. This is version 1, last updated on . Download it on this page, a zip of its files. Nothing in it runs by itself.
pip install -U huggingface_hub hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed
Qwen3.8-27B-ASCII-Condensed is a GGUF build of Qwen3.8-27B whose vocabulary keeps only ASCII tokens, so more of a 16 GB graphics card is left for long context. On an RTX 5070 Ti the author reached a context of 135,168 tokens, while the same quantization without the cut manages 114,688. No weights were retrained or requantized.
It suits people who run large local models with llama.cpp on one 16 GB graphics card and need a long prompt window for English text or code. Anyone who works mostly in other languages should look elsewhere.
| Industry | Who uses it | How they use it | Example |
|---|---|---|---|
| Software development | Developers running a local coding assistant | Feed long files into the prompt | Up to 135,168 tokens fit on a 16 GB card in the author's tests |
| Research and writing | Analysts who work in English | Question long documents on their own machine | A long document stays in context on a single card |
| Local AI hobby | Enthusiasts with one consumer GPU | Run a 27B model at long context without a larger card | An RTX 5070 Ti reached 135,168 tokens |
| Use case | Who | What they get back |
|---|---|---|
| Long-prompt chat | Local model users | A usable context of up to 135,168 tokens on a 16 GB card |
| Faster decoding with MTP | Users who accept a shorter context | About 1.8x faster decoding at up to 77,824 tokens |
| Code and English text work | Developers | The same tokenizer output as the base model on ASCII text |
You need llama.cpp and a graphics card with 16 GB of memory.
pip install -U huggingface_hub hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed
The quickstart command from the card:
llama-cli -m Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf \ -ngl 99 -fa on -ctk q4_0 -ctv q4_0 \ -ot "token_embd.weight=CPU" \ -c 135168 -ub 128 -b 512
Run the quickstart command, then chat or prompt as with any llama.cpp model. The embedding table stays in system RAM, which costs about 1% of decode speed and frees its whole footprint on the card.
| Tool | What it does |
|---|---|
| -ot token_embd.weight=CPU | Keeps the embedding table in system RAM |
| -ctk and -ctv | Set the cache types, which must match on stock builds |
| -c | Sets the context size, 135168 in the quickstart |
| --spec-type draft-mtp | Turns on MTP speculative decoding, about 1.8x faster at a smaller context |
For MTP, add these flags:
--spec-type draft-mtp --spec-draft-n-max 3 -ctkd q4_0 -ctvd q4_0
You run the quickstart command on a 16 GB card and send a prompt about 15K tokens deep. The author measured decoding near 47.3 tokens per second there, and near 33.1 at 90K, because decoding slows as the cache fills.
Think of it as the same 27B model with the unused part of its dictionary removed, which leaves room for a longer conversation. The first thing to try is the quickstart command with a short prompt, then a longer one.
Underneath, non-ASCII rows were removed from the token embedding and output tensors, and the tokenizer was rewritten to match. The vocabulary shrank from 248,320 to 129,006 rows, and 864 of 866 tensors match the base quant bit for bit. Embedding is a gather and the output head runs over the whole vocabulary for every token, so shrinking both frees VRAM for the KV cache. The trade-off is that non-ASCII text costs extra tokens.
Context ceilings depend on the cache types you choose, and they were measured on a 16 GB card with a real prompt of about 15K tokens.
| Prompt depth | Prefill | Decode |
|---|---|---|
| 15K | 1,501 t/s | 47.3 t/s |
| 90K | 930 t/s | 33.1 t/s |
| Text | Cost |
|---|---|
| English, code, ASCII punctuation | Unchanged |
| Unaccented Latin script | Unchanged |
| Accented Latin letters | About 2 tokens per accented character |
| CJK, Cyrillic, Arabic, Thai | About 3 tokens per character |
Runs Qwen3.8-27B with a vocabulary cut down to ASCII, so a single 16 GB graphics card can hold a much longer prompt. A GGUF build for English text and code with llama.cpp.
Made by bsaleh03 on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.
What it is: Qwen3.8-27B-ASCII-Condensed is a GGUF build of Qwen3.8-27B whose vocabulary keeps only ASCII tokens, so more of a 16 GB graphics card is left for long context. On an RTX 5070 Ti the author reached a context of 135,168 tokens, while the same quantization without the cut manages 114,688. No weights were retrained or requantized.
Who it suits: It suits people who run large local models with llama.cpp on one 16 GB graphics card and need a long prompt window for English text or code. Anyone who works mostly in other languages should look elsewhere.
Industry | Who uses it | How they use it | Example
Software development | Developers running a local coding assistant | Feed long files into the prompt | Up to 135,168 tokens fit on a 16 GB card in the author's tests
Research and writing | Analysts who work in English | Question long documents on their own machine | A long document stays in context on a single card
Local AI hobby | Enthusiasts with one consumer GPU | Run a 27B model at long context without a larger card | An RTX 5070 Ti reached 135,168 tokens
Where to use it:
Use case | Who | What they get back
Long-prompt chat | Local model users | A usable context of up to 135,168 tokens on a 16 GB card
Faster decoding with MTP | Users who accept a shorter context | About 1.8x faster decoding at up to 77,824 tokens
Code and English text work | Developers | The same tokenizer output as the base model on ASCII text
How to set it up: You need llama.cpp and a graphics card with 16 GB of memory.
pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed
The quickstart command from the card:
llama-cli -m Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf \
-ngl 99 -fa on -ctk q4_0 -ctv q4_0 \
-ot "token_embd.weight=CPU" \
-c 135168 -ub 128 -b 512
How to use it: Run the quickstart command, then chat or prompt as with any llama.cpp model. The embedding table stays in system RAM, which costs about 1% of decode speed and frees its whole footprint on the card.
Tool | What it does
-ot token_embd.weight=CPU | Keeps the embedding table in system RAM
-ctk and -ctv | Set the cache types, which must match on stock builds
-c | Sets the context size, 135168 in the quickstart
--spec-type draft-mtp | Turns on MTP speculative decoding, about 1.8x faster at a smaller context
For MTP, add these flags:
--spec-type draft-mtp --spec-draft-n-max 3 -ctkd q4_0 -ctvd q4_0
Example: You run the quickstart command on a 16 GB card and send a prompt about 15K tokens deep. The author measured decoding near 47.3 tokens per second there, and near 33.1 at 90K, because decoding slows as the cache fills.
For newcomers and experts: Think of it as the same 27B model with the unused part of its dictionary removed, which leaves room for a longer conversation. The first thing to try is the quickstart command with a short prompt, then a longer one.
Underneath, non-ASCII rows were removed from the token embedding and output tensors, and the tokenizer was rewritten to match. The vocabulary shrank from 248,320 to 129,006 rows, and 864 of 866 tensors match the base quant bit for bit. Embedding is a gather and the output head runs over the whole vocabulary for every token, so shrinking both frees VRAM for the KV cache. The trade-off is that non-ASCII text costs extra tokens.
Start from the UD-IQ4_XS quantization
Drop non-ASCII rows from two tensors
Rewrite the tokenizer to match
Freed VRAM becomes KV cache
Longer context fits on a 16 GB card
Good to know: Context ceilings depend on the cache types you choose, and they were measured on a 16 GB card with a real prompt of about 15K tokens.
Title: Max context by cache type
q4_0 + q4_0: 135168 tokens
q5_1 + q4_0: 118784 tokens
q8_0 + q4_0: 98304 tokens
q8_0 + q8_0: 77824 tokens
Title: Speed on an RTX 5070 Ti at q4_0 and q4_0
Prompt depth | Prefill | Decode
15K | 1,501 t/s | 47.3 t/s
90K | 930 t/s | 33.1 t/s
Title: Token cost of non-ASCII text
Text | Cost
English, code, ASCII punctuation | Unchanged
Unaccented Latin script | Unchanged
Accented Latin letters | About 2 tokens per accented character
CJK, Cyrillic, Arabic, Thai | About 3 tokens per character
Download its files with Hugging Face's command line:
pip install -U huggingface_hub
hf download bsaleh03/Qwen3.8-27B-ASCII-Condensed
Then follow the usage in its model card. Nothing in this kit runs by itself.
README.md 7 KB
Listed Hugging Face models in the same category that share the most tags with this one.
| Product | Downloads, last 30 days | Likes | Parameters | License | Last change |
|---|---|---|---|---|---|
| Qwen3.8-27B-ASCII-Condensed This page | 1,677 | 35 | 26.1B | apache-2.0 | |
| Sharp-Spark-X2.5-4B-GGUFShares long-context · imatrix · llama.cpp | 875,346 | 164 | 4.1B | apache-2.0 | |
| Sharp-Spark-X2.6-4B-GGUFShares long-context · imatrix · llama.cpp | 5,044 | 23 | 4.1B | apache-2.0 | |
| Qwen3.8-27B-Ridge-GGUFShares long-context · imatrix · llama.cpp | 292,932 | 369 | 27.3B | apache-2.0 | |
| MiniCPM5-2B-GGUFShares long-context · llama.cpp · gguf | 481,119 | 393 | 2.5B | apache-2.0 | |
| Qwen3-30B-A3B-GGUF by unslothShares qwen3 · imatrix · gguf | 53,664 | 294 | 30.5B | apache-2.0 |
sentence-transformers · Kit
Turns sentences and paragraphs into 768-number vectors so you can search, cluster and compare text by meaning. It suits developers and data teams working in En…
intfloat · Kit
Turns text in about 100 languages into vectors so you can compare meaning across them, for search and matching. Made for developers who build multilingual retr…
aldegad · Skill
Turns one base drawing of a character into game-ready sprites, either an atlas with a manifest or transparent motion loops, for game developers working in Code…
foryourhealth111-pixel · Skill
Picks the right skills from a large installed pool and breaks each task into checkable steps for your coding agent. In its own benchmark, mean reward rose 21.1…
elementalsouls · Skill
Gives Claude Code 83 skills and 15 slash commands for bug bounty hunting and authorized red-team work, from web vulnerability classes to enterprise identity at…
axteam · Kit
Four GGUF quantizations of GLM-4.7-Flash for running a 31B-class chat model locally in llama.cpp, from 18.9 GB to 33.2 GB. For people who host models on their…