Raptor-0.6-4B-JANG_6M

Raptor-0.6-4B-JANG_6M

Decides when to call a tool, when to answer directly and when to ask first, tuned from a 4B model and quantized for Apple Silicon Macs running Osaurus. For people who run local assistants.

Made by OsaurusAI on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: Raptor 0.6 is a text generation model from OsaurusAI. The team trained it further from the 4B base model XHToken/Spark-X2.5-4B for the Osaurus app. Its training covered choosing between a tool call and a plain answer, picking the right tool, recovering after errors, asking the user, and passing work to sub-agents. This JANG_6M bundle is a mixed-precision quantization that takes 3.41 GiB on disk, according to the card.

Who it suits: It suits people with an Apple Silicon Mac who want a small local model that handles tool use inside Osaurus, and developers who study how such models are tuned and measured.

Industry | Who uses it | How they use it | Example
Software | App developers | Run a small local model as the decision maker in an assistant | Choosing between answering directly and calling a tool
Productivity | Mac users with local assistants | Let Osaurus handle everyday requests on their own machine | Asking the assistant to schedule or look up something
Research | Quantization researchers | Study calibrated mixed-precision quantization of a 4B model | Reading the KL and top-1 agreement figures against the bf16 weights

You can skip it if you do not use Osaurus or an Apple Silicon Mac, since the card says stock mlx-lm cannot run this architecture.

Where to use it:

• The Osaurus app on an Apple Silicon Mac, which ships the MLX runtime this model needs
• Python on Windows, macOS or Linux, only to download the files

Use case | Who | What they get back
Tool or no tool | Assistant users | A direct answer, or a call to the right tool
Recovering from errors | App developers | A retry or a change of approach after a failed call
Delegating work | Agent builders | Work handed on to sub-agents

How to set it up:

• An Apple Silicon Mac with the Osaurus app
• Python and pip, to fetch the files by hand
• About 3.41 GiB of disk space, as the card states

1. Install the Hugging Face hub package.
2. Download the model files to your computer.
3. Open the model card and follow its steps to load the model in Osaurus.

pip install -U huggingface_hub
hf download OsaurusAI/Raptor-0.6-4B-JANG_6M

How to use it: Pick the model inside Osaurus and chat as usual, with reasoning turned on or off. The tune covers both modes, and the tokenizer and chat template are the same as the base model's. The repository carries a version marker of 2 in its osaurus.json file, which lets Osaurus flag an older local copy.

Tool | What it does
huggingface_hub | The Hugging Face download package
hf download | Copies the model files to your computer

Example: You ask the assistant to clear out old files in a folder. The card lists confirming before a destructive step among the training topics, so this is the kind of case the model was tuned for, and you can expect it to ask first.

For newcomers and experts: It is a small model that learned to judge when a request needs a tool and when a plain answer is enough. To try it, download the files with the two commands above, then pick the model in Osaurus on a Mac.

Iteration 2 is a LoRA fine-tune of rank 16 and alpha 32, applied to the attention and MLP layers, with 180 matrices folded back into the weights. The training set holds about 3.5k decisions, including replies the base model already got right. Attention and the tied embedding are stored with 8 bits and the MLP with 6 bits, in groups of 64 weights, which averages 7.126 bits per weight. Norms and the per-head attention gate stay in full precision. Calibration lowers the KL by roughly 2.6 to 2.9 times at the same size. On an M5 Max the card reports 105.4 tokens per second for decoding and 5042 for prefill.

Base model Spark-X2.5-4B
LoRA fine-tune on tool decisions
Calibrated mixed-precision quantization
Runs on Apple Silicon through Osaurus

Good to know: The card measured the quantization against the model's own bf16 weights on two held-out sets, and lower KL is better.

Title: Mean KL against bf16 weights, lower is better
Tool conversations, this bundle: 0.0140 nats
Tool conversations, uncalibrated rounding: 0.0402 nats
Mixed prompts, this bundle: 0.0102 nats
Mixed prompts, uncalibrated rounding: 0.0268 nats

• License: Apache 2.0
• Cost to run: no API key or paid service is named, but the Osaurus app and a Mac are needed, and the source gives no price for Osaurus
• Data it reaches: the source names no files, accounts or web access beyond downloading the weights from Hugging Face
• Not for: stock mlx-lm, which has no spark2_5 architecture, or the older Raptor v0.5 line, whose runtime assumptions do not carry over

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Parameters: 4.1B
• Weights: about 16.4 GB in U32
• Formats: Safetensors
• Downloads in the last 30 days: 1,032
• Likes: 7
• License: apache-2.0
• Languages: en, zh
• Based on: XHToken/Spark-X2.5-4B
• Tags: safetensors, spark2_5, jang, quantized, apple-silicon, reasoning, agent, tool-use, imatrix, gptq, text-generation, conversational

Links

• Model: https://huggingface.co/OsaurusAI/Raptor-0.6-4B-JANG_6M
• Files: https://huggingface.co/OsaurusAI/Raptor-0.6-4B-JANG_6M/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download OsaurusAI/Raptor-0.6-4B-JANG_6M

Then follow the usage in its model card. Nothing in this kit runs by itself.