Parallel Constrained Decoding for Apple Silicon

Parallel Constrained Decoding for Apple Silicon

Extracts structured fields, routes decisions and classifies text on Apple Silicon with MLX, scoring every schema field at once. Cuts latency 5.6x to 7.0x on an M4 Max, with always-valid JSON.

Made by harshatheg on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is an inference engine for structured extraction, decision routing and classification on Apple Silicon, built on MLX. Instead of writing a JSON answer token by token, it scores every schema field in parallel. The card reports 5.6x to 7.0x lower latency on an M4 Max, with 100% schema validity and a confidence score for each field.

Who it suits: It suits developers on Macs who turn text into fixed fields, such as a priority level or a yes or no flag. The card's own cases include fraud routing, code security audits, tariff choice and support triage.

Title: Who runs it
Industry | Who uses it | How they use it | Example
Fintech | Risk teams | Route transactions by a few fixed fields | Fraud routing on 4 fields
Support | Support operations teams | Triage tickets into priority and escalation | A ticket triage with 28 fields
Software | Security engineers | Label code for audit with set categories | A code security audit on 4 fields

If you do not use an Apple Silicon Mac, you can skip it. It needs MLX.

Where to use it:

• An Apple Silicon Mac, from the M1 to the M4 series
• macOS 14.0 or later, with Python 3.10 or newer
• Python code that defines a schema of enum and boolean fields

Title: Jobs it does
Use case | Who | What they get back
Structured extraction | Developers | Valid JSON built from verified field values
Decision routing | Operations teams | One chosen category per field with a confidence score
Classification with many choices | Data teams | A pick among up to 255 choices

How to set it up:

• An Apple Silicon Mac running macOS 14.0 or later
• Python 3.10 or newer
• The project's repository cloned to your machine

1. Clone the project repository and enter its folder.
2. Create a virtual environment and activate it.
3. Install the requirements.

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

pip install -U huggingface_hub
hf download harshatheg/Qwen-2.5-1B-RLCD

How to use it: Define a schema with StructuredSchema, where each field is an enum or a boolean with a description and, for enums, up to 255 choices. Give the engine your text, and it fills all fields at once and returns JSON with a confidence score per field. The card's benchmarks used a 4-bit Qwen2.5-1.5B-Instruct model.

Title: Pieces the card names
Tool | What it does
StructuredSchema | Defines the fields of the output
FieldDefinition | Describes one field, its type and its choices
enum | A field with a fixed list of choices, up to 255
boolean | A yes or no field

Example: You define two fields for a support ticket: a priority with four levels, and a yes or no field for escalation. You pass a customer message. The engine returns JSON with both fields filled and a confidence score for each, without writing the JSON token by token.

For newcomers and experts: If you are new, normal models write an answer one piece at a time, which is slow for forms with many fields. This engine reads the text once and fills every field at the same time. The first small thing to try is a schema with one yes or no field.

If you know the field, it prefills the context once into an MLX KV cache and broadcasts it across the fields. For each field it scores only the token ids of valid choices and normalizes a softmax over that slice. It resolves shared prefixes with a token tree and builds the JSON in code. The trade-off is scope: it suits bounded choices, not free text.

Prefill the context once
Broadcast the cache to every field
Score only the valid choices per field
Build the JSON from verified values

Good to know:

Title: Latency speedup over standard decoding
Fintech Fraud Routing: 5.6 times
Code Security Audit: 5.6 times
High-Cardinality Tariff: 5.6 times
Enterprise Support Triage: 7.0 times

• Setting: the numbers come from an Apple Silicon M4 Max, and results on other chips may differ.
• Validity: the card says the JSON is always valid because it is built in code from checked values.
• Limits: fields hold booleans or choices from a fixed list, not free text.
• Demo: the card links a live side-by-side comparison on Hugging Face Spaces.
• License: apache-2.0.
• Cost to run: no API key, account or paid service is named.
• Data it reaches: everything runs on your Mac, and you download files from Hugging Face.
• Not for: free-text answers and machines without Apple Silicon.

On Hugging Face, 13 November 2026

• Task: text generation
• Library: mlx
• Downloads in the last 30 days: 0
• Likes: 592
• License: apache-2.0
• Languages: en
• Based on: Qwen/Qwen2.5-1.5B-Instruct
• Tags: structured-generation, parallel-decoding, constrained-decoding, apple-silicon, classification, json, text-generation

Links

• Model: https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD
• Files: https://huggingface.co/harshatheg/Qwen-2.5-1B-RLCD/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download harshatheg/Qwen-2.5-1B-RLCD

Then follow the usage in its model card. Nothing in this kit runs by itself.