Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark

Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark

Serves the 122B Ornith 1.5 model through vLLM with DFlash2 speculative decoding, using an unquantized output head. Made for RTX Spark laptops with 128 GB of memory.

Made by Sherlock-Comms on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is the Ornith-1.5-122B-NVFP4 checkpoint with one change. Its lm_head is stored in BF16 instead of NVFP4. vLLM only accepts the DFlash2 draft model when that head is unquantized, and the official checkpoint quantizes it. The author tested it on a 128 GB RTX Spark N1X laptop and reports that other GPUs running vLLM work too.

Who it suits: It suits people who serve huge models on their own hardware and want faster decoding from a draft model. The author says it has no connection to Ornith AI, and none to Inco AI.

Industry | Who uses it | How they use it | Example
Local AI | Owners of RTX Spark N1X laptops with 128 GB | Serve a 122B model with vLLM and speculative decoding | Running a long agent conversation on a laptop
Software development | Agent builders | Use tool calling and long context from a local server | Keeping a 61K-token conversation going across several turns
Inference engineering | vLLM users on other GPUs | Try DFlash2 drafting with a target whose head is unquantized | Checking whether the draft model is accepted

You can skip it if you cannot give the GPU about 96 GB, which the card reports using, or if you want a model with vendor support.

Where to use it:

• vLLM 0.32.0, the version the card tests
• Docker under WSL 2 on Windows 11 on Arm, as in the card's test setup
• Other GPUs running vLLM, which the card says also work

Use case | Who | What they get back
Long-context chat | Local AI users | A server that accepts requests up to 262,144 tokens
Agent work with tools | Agent builders | Tool calls parsed with the qwen3_coder parser
Image questions | Developers | Answers about up to 4 images per prompt, with no video

How to set it up:

• Python, to run the Hugging Face download tool
• The huggingface_hub package
• vLLM 0.32.0, with Docker under WSL 2 if you copy the tested setup
• A GPU with about 96 GB of memory available, as on the 128 GB test laptop
• Room for about 78.7 GB of model files

1. Install the Hugging Face hub package.
2. Download the repository files to your computer.
3. Start vLLM with the card's flags, putting this repository's name where the placeholder is.

pip install -U huggingface_hub
hf download Sherlock-Comms/Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark

vllm serve <this-repo> --served-model-name ornith-1.5-122b \
  --max-model-len 262144 --max-num-seqs 1 --max-num-batched-tokens 8192 \
  --kv-cache-dtype fp8 --kv-cache-memory-bytes 12884901888 \
  --enable-prefix-caching --prefix-cache-retention-interval 17344 \
  --speculative-config '{"method":"dflash","model":"incoai/Ornith-1.5-122B-DFlash2","num_speculative_tokens":7}' \
  --moe-backend marlin --no-async-scheduling \
  --mm-encoder-attn-backend TORCH_SDPA --limit-mm-per-prompt '{"image":4,"video":0}' \
  --mm-processor-cache-gb 1 --skip-mm-profiling \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --trust-remote-code

How to use it: Once the server is up, you talk to the model under the served name ornith-1.5-122b. The flags matter for this model, and the card explains several of them.

Tool | What it does
--prefix-cache-retention-interval | Needed for prefix caching to get hits. 17344 fits 7 draft tokens, while 8640 fits 6 and 17472 fits 8
--no-async-scheduling | Required because async scheduling does not work with DFlash
--moe-backend marlin | Keeps vLLM off a kernel that hung the GPU on this chip
--skip-mm-profiling | Saves about 13 s at startup, and vision still works
TORCH_SDPA | The vision encoder backend, since flash-attn has no sm_121 build
--kv-cache-memory-bytes | Sets about 12 GiB of FP8 cache, enough for a full 262,144-token request

Example: You keep one agent conversation going, with the start of the prompt unchanged. After the first turn the server reuses the cached prefix. In the card's 61K-token test, answers began in about 9 s instead of 38 to 41 s.

For newcomers and experts: This page is for a narrow setup, a huge model on a laptop with a lot of memory. To start, run the two download commands, then read the card's flag list before you launch the server.

If you know the field, the BF16 head was dequantized from the official NVFP4 head, so outputs match the official model. Sampled rows match the original BF16 head at cosine 0.995. The expert layers keep W4A16 NVFP4 weights, which sends vLLM to its Marlin kernel.

The official NVFP4 head is dequantized to BF16
vLLM now accepts the DFlash2 draft
The draft proposes 7 tokens each step
The 122B model checks them, and about 3 are kept

Good to know:

Title: Decode speed in tokens per second
This checkpoint with DFlash2: 50 tok/s
Q4_K_M GGUF with MTP: 36.6 tok/s

• Memory: the test laptop's GPU memory was set to 96 GB, with about 94 GB used and a peak near 97 GB at startup
• Tool calling: 53 of 54 runs passed in the author's test, with no invalid calls
• Size: about 78.7 GB, up from 77.6 GB for the official checkpoint
• Results come from one laptop, an ASUS ProArt P16
• License: mit
• Cost to run: none named, but the setup needs a GPU with about 96 GB of memory
• Data it reaches: the prompts and images you send, the draft model named in the command, and the files you download from Hugging Face
• Not for: official support, since the author says it has no connection to Ornith AI or Inco AI

On Hugging Face, 13 November 2026

• Task: image text to text
• Library: vllm
• Parameters: 64B
• Weights: about 64 GB in U8
• Formats: Safetensors
• Downloads in the last 30 days: 342
• Likes: 2
• License: mit
• Based on: ornith-ai/Ornith-1.5-122B-NVFP4
• Tags: vllm, safetensors, qwen3_5_moe, nvfp4, modelopt, dflash, dflash2, speculative-decoding, rtx-spark, n1x, image-text-to-text, conversational

Links

• Model: https://huggingface.co/Sherlock-Comms/Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark
• Files: https://huggingface.co/Sherlock-Comms/Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download Sherlock-Comms/Ornith-1.5-122B-NVFP4-DFlash2-RTX-Spark

Then follow the usage in its model card. Nothing in this kit runs by itself.