Spark-X2.6-4B-DSpark-GGUF

Spark-X2.6-4B-DSpark-GGUF

Speeds up text generation for Spark-X2.6-4B as a small draft model for speculative decoding, in GGUF format. Meant for people running that model locally with llama.cpp.

Made by XHToken on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a draft model for speculative decoding, a BF16 GGUF file of 624M parameters made by XHToken. It pairs with the Spark-X2.6-4B model. The draft guesses several tokens ahead, and the main model checks those guesses together, so fewer slow one-token steps are needed.

Who it suits: People who already run Spark-X2.6-4B on their own computer and want faster replies, plus developers who study speculative decoding.

Industry | Who uses it | How they use it | Example
Software | Developers | Serve Spark-X2.6-4B locally with a draft model attached | A local chat endpoint that answers faster
Research | ML engineers | Measure how many drafted tokens the main model accepts | Compare acceptance on math and coding sets
Edge computing | On-device builders | Raise token speed on a single GPU or a Mac | A laptop assistant that replies sooner

If you do not run Spark-X2.6-4B, you can skip it, since the draft is of no use alone.

Where to use it:

• A llama.cpp build from XHToken, which the card uses for its steps
• Linux with Bash and an NVIDIA GPU, for the card's own commands
• Apple silicon through MLX, where the card reports its own speed tests

Use case | Who | What they get back
Faster local chat | Developers | Replies generated with fewer sequential steps
OpenAI-style server | App builders | A local endpoint that speaks a familiar chat format
Speed testing | Researchers | Acceptance and throughput figures for drafts

How to set it up:

• Python, to run the Hugging Face download tool
• The Spark-X2.6-4B target model in GGUF form, in BF16 or quantized weights
• A CUDA build of the card's llama.cpp fork, since upstream support is still in progress

1. Download this draft model with the hf command.
2. Download the Spark-X2.6-4B target model as well.
3. Build the llama.cpp fork the card names, with CUDA turned on.
4. Start llama-server with the target as the main model and this file as the draft.

pip install -U huggingface_hub
hf download XHToken/Spark-X2.6-4B-DSpark-GGUF

How to use it: Start the server with both GGUF files, then send chat requests to it like you would to any OpenAI-style endpoint. The card's start command is below, with the draft limited to seven tokens per pass.

set -o pipefail
./build-cuda/bin/llama-server \
  -m models/Spark-target.gguf \
  -md models/Spark-draft.gguf \
  --alias spark-dspark \
  --spec-type draft-dspark \
  --spec-draft-sampling greedy \
  --spec-draft-n-max 7 \
  --spec-draft-p-min 0 \
  -ngl 999 -ngld 999 \
  -ctk f16 -ctv f16 \
  -ctkd f16 -ctvd f16 \
  -fa on \
  -c 8192 -np 1 \
  -b 2048 -ub 512 -t 8 \
  --jinja \
  --host 127.0.0.1 --port 8080 \
  2>&1 | tee dspark-server.log

Tool | What it does
-m | Sets the target model file
-md | Sets the draft model file
--spec-draft-n-max | Caps how many tokens the draft proposes per pass
--alias | Names the model for requests, here spark-dspark

Example: With the server running, you send a chat request whose user message is "你好" and ask for at most 512 tokens. The server returns a normal chat reply, produced with the draft model helping behind the scenes.

For newcomers and experts: Start by getting Spark-X2.6-4B running on its own, then add this file as the draft and compare the speed. Keep the first test short.

The draft has 5 sliding-window attention layers with a window of 512 and a hidden size of 2560. It reads hidden states from five target layers. The card reports average acceptance lengths of about 4.0 to 4.6 out of a maximum of 8, depending on the thinking mode.

Target model is loaded
Draft proposes up to 7 tokens
Target checks them in parallel
Accepted tokens are kept

Good to know:

• License: Apache 2.0
• Cost to run: none named, though you need an NVIDIA GPU for the card's commands
• Data it reaches: the card's server listens on your own machine, and the download comes from Hugging Face
• Not for: use without its Spark-X2.6-4B target model
• Not for: the card's own commands on systems other than Linux, Bash and NVIDIA GPUs

Title: Average speedup in the card's on-device tests
BF16 on RTX 4090: 2.14 times
BF16 on M5 Pro: 2.50 times
Q4_K_M on RTX 4090: 1.98 times
INT4 on M5 Pro: 1.62 times

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 624M
• Weights: about 312 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 4,008
• Likes: 29
• License: apache-2.0
• Languages: en, zh
• Based on: XHToken/Spark-X2.6-4B
• Tags: gguf, sparkx2_6, dspark, speculative-decoding, draft-model, llama-cpp, text-generation, conversational

Links

• Model: https://huggingface.co/XHToken/Spark-X2.6-4B-DSpark-GGUF
• Files: https://huggingface.co/XHToken/Spark-X2.6-4B-DSpark-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download XHToken/Spark-X2.6-4B-DSpark-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.