Spark-X2.6-1.7B-DSpark-GGUF

Spark-X2.6-1.7B-DSpark-GGUF

A small draft model that speeds up Spark-X2.6-1.7B through speculative decoding, guessing up to seven tokens per step. For people who run that model locally on a llama.cpp build that supports it.

Made by XHToken on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a small draft model in BF16 GGUF format, made to speed up Spark-X2.6-1.7B through speculative decoding. The small model guesses up to seven tokens at once, and the main model checks the guesses together. That means fewer one-by-one steps for the main model. The main model can be in BF16 or in a quantized GGUF.

Who it suits: People who already run Spark-X2.6-1.7B locally and want faster replies, and researchers who study speculative decoding.

Industry | Who uses it | How they use it | Example
Software | Developers with a local model | Pair the draft with the main model in a server | Serve a faster local chat endpoint
Local AI | Hobbyists with an NVIDIA GPU | Raise tokens per second on one card | Run a math or coding prompt at higher speed
Research | Inference researchers | Study how many drafted tokens get accepted | Compare acceptance in thinking and non-thinking modes

You can skip it if you do not run Spark-X2.6-1.7B, since the draft only helps that one target model.

Where to use it:

• XHToken's llama.cpp fork, which serves an OpenAI-compatible chat API
• Any tool that can call an OpenAI-compatible chat endpoint
• Linux with a Bash shell and an NVIDIA GPU, which is what the card's commands target

Use case | Who | What they get back
Faster local chat | App developers | The same style of reply, produced with fewer main-model steps
Benchmarking | Researchers | Acceptance length and tokens per second with and without the draft
On-device agents | Builders of small agents | Quicker generation from a 1.7B model

How to set it up: You need the fork of llama.cpp, a build with CUDA, the main model file and this draft file. The card says support in the main llama.cpp project is still in progress. The sample steps assume a Linux machine with an NVIDIA card.

• Git, CMake and a CUDA toolchain
• An NVIDIA GPU
• Python and pip, to download the files
• The main model, Spark-X2.6-1.7B-GGUF, as well as this draft

1. Clone XHToken's llama.cpp fork and move into the folder.
2. Build llama-server with CUDA on, using the commands below.
3. Download both model files into a models folder.

cd llama.cpp
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --target llama-server -j8

To fetch this repository's files yourself, Hugging Face gives these steps:

pip install -U huggingface_hub
hf download XHToken/Spark-X2.6-1.7B-DSpark-GGUF

How to use it: Start llama-server with the main model and this draft, then send chat requests to its OpenAI-compatible route on port 8080. The card names the files Spark-target.gguf and Spark-draft.gguf in its command.

set -o pipefail
./build-cuda/bin/llama-server \
  -m models/Spark-target.gguf \
  -md models/Spark-draft.gguf \
  --alias spark-dspark \
  --spec-type draft-dspark \
  --spec-draft-sampling greedy \
  --spec-draft-n-max 7 \
  --spec-draft-p-min 0 \
  -ngl 999 -ngld 999 \
  -ctk f16 -ctv f16 \
  -ctkd f16 -ctvd f16 \
  -fa on \
  -c 8192 -np 1 \
  -b 2048 -ub 512 -t 8 \
  --jinja \
  --host 127.0.0.1 --port 8080 \
  2>&1 | tee dspark-server.log

Option | What it does
-m | Loads the main Spark-X2.6-1.7B model
-md | Loads this draft model
--spec-type draft-dspark | Turns on DSpark drafting
--spec-draft-n-max 7 | Lets the draft propose up to seven tokens per pass
--alias spark-dspark | Sets the model name that requests use

Example: With the server running, you send a chat request with the model name spark-dspark and one user message, "你好". You set temperature 1.0, top_p 0.95 and max_tokens 512, with streaming off. The server returns a normal chat completion. The card shows no sample reply.

For newcomers and experts: If you are new, remember that this file is a helper and cannot chat on its own. Run it only next to the main Spark model, and start with the one server command above.

For experts, the draft has 5 sliding-window layers with a 512-token window and a hidden size of 2048. It reads hidden states from target layers 2, 8, 14, 19 and 25 and drafts 7 tokens per forward pass. The acceptance length counts one extra target token, so 8 is the top value. The speed-up shrinks when the main model is quantized.

Prompt reaches the server
Draft model guesses up to 7 tokens
Main model checks them together
Accepted tokens are returned

Good to know:

Title: Average tokens per second on an RTX 4090, from the card's own tests
BF16 alone: 220 tok/s
BF16 with DSpark: 463 tok/s
Q4_K_M alone: 424 tok/s
Q4_K_M with DSpark: 650 tok/s

• Results: the speed-ups come from the model authors' own tests on an RTX 4090 with llama.cpp and an Apple M5 Pro with MLX
• Throughput: on the RTX 4090 with a BF16 target, the average went from 220 to 463 tokens per second
• Languages: English and Chinese
• Draft size: 345M parameters
• License: Apache 2.0
• Cost to run: no API key or paid service named
• Data it reaches: downloads files from Hugging Face and the llama.cpp fork, and the sample server listens on 127.0.0.1 only
• Not for: use on its own, or with a different main model than Spark-X2.6-1.7B

On Hugging Face, 13 November 2026

• Task: text generation
• Library: gguf
• Parameters: 345M
• Weights: about 173 MB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 969
• Likes: 10
• License: apache-2.0
• Languages: en, zh
• Based on: XHToken/Spark-X2.6-1.7B
• Tags: gguf, sparkx2_6, dspark, speculative-decoding, draft-model, llama-cpp, text-generation, conversational

Links

• Model: https://huggingface.co/XHToken/Spark-X2.6-1.7B-DSpark-GGUF
• Files: https://huggingface.co/XHToken/Spark-X2.6-1.7B-DSpark-GGUF/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download XHToken/Spark-X2.6-1.7B-DSpark-GGUF

Then follow the usage in its model card. Nothing in this kit runs by itself.