pplx-decider-flash-gguf-v1.2

pplx-decider-flash-gguf-v1.2

Sorts a request into the choices you set, from text and images, using a Q8_0 GGUF copy of a small decision model. For developers who can run a protocol v3 runtime.

Made by perplexity-ai on Hugging Face. These notes are Mindket's, written from its model page and model card; the model's own files and license are on Hugging Face.

About

What it is: This is a GGUF copy of pplx-decider-flash-v1.2, a small text classification model from perplexity-ai. It reads a request in text, with images if you add them, and answers structured questions by scoring the options you give. The model is built on Qwen3.5-4B and was distilled from larger 27B decision models.

Who it suits: Developers who sort or label requests and want a small model on their own hardware. It needs a runtime that supports the model's own protocol, so it fits people who can build or adapt one.

Industry | Who uses it | How they use it | Example
Customer support | Helpdesk tool developers | Route incoming tickets to a team | A ticket about a failed API call goes to support
Software teams | Platform engineers | Label requests with fixed options inside an app | Pick a category and a yes or no for urgency
Applied AI | Researchers and tinkerers | Try a cheaper decision model on text and image tasks | Compare it with the larger 27B model

Anyone who wants a chat assistant or text generation can skip it, because this model only picks among options.

Where to use it:

• Windows, macOS or Linux: the download steps run on all three with Python.
• A compatible runtime: any program that implements protocol v3, since stock llama-server does not.
• Apple Metal: the maker checked the Q8_0 file there against a float32 reference.
• Hugging Face: the files are hosted there for download.

Use case | Who | What they get back
Ticket routing | Support tool developers | A probability for each team option, such as billing, support or sales
Urgency checks | App developers | A probability for each answer to a yes or no question about the request
Requests with images | Developers with mixed input | Scores for options that depend on both the text and the attached image

How to set it up: What you need first:

• Python with pip, on Windows, macOS or Linux.
• Disk space for two files, 5.16 GB and 672 MB.
• A runtime that implements protocol v3, because stock llama-server does not.

1. Install the Hugging Face hub package with pip.
2. Download this repository with the hf command.
3. Load the files in a runtime that builds the joint prompt and reads the option scores.

pip install -U huggingface_hub
hf download perplexity-ai/pplx-decider-flash-gguf-v1.2

The base model card says its own repository is private and needs a Hugging Face token with access. Its Python path needs Python 3.12 or newer and a CUDA GPU.

How to use it: You send one request, called a state, plus a set of questions. Each question has instructions and criteria that describe its options. The model returns a probability for each option. The Python below comes from the base model card and loads the base checkpoint, not these GGUF files.

import sys
from pathlib import Path
from huggingface_hub import snapshot_download

checkpoint = Path(snapshot_download("perplexity-ai/pplx-decider-flash-v1.2"))
sys.path.insert(0, str(checkpoint / "source" / "src"))
from pplx_decider import Decider

decider = Decider.from_pretrained(checkpoint, device="cuda")

Title: Files and calls named in the source
Tool | What it does
decider-flash-v1.2-q8_0.gguf | The 8-bit main weights plus the part that scores options, 5.16 GB
mmproj-decider-flash-v1.2-f16.gguf | F16 vision projector for image input, 672 MB
Decider.from_pretrained | Loads the base checkpoint onto a CUDA device in the Python path
decide | Takes a state and its questions and returns a response

Example: A customer writes that their integration returns a 401 error since the API key was changed. You ask which team should handle it, with billing, support and sales as the options. The model returns a probability for each option, and you act on the highest one.

For newcomers and experts: Think of it as a small sorting helper. You hand it a message and a question with a few possible answers, and it tells you how likely each answer is. A good first step is to run the two download commands and read the model card to see how questions are written.

The student model learned the teachers' full probability distributions instead of only their answers, and several training runs were merged by weight averaging. In this GGUF the readout sits in output.weight, zero outside the code tokens. Full-attention layers are non-causal while linear-attention layers stay causal. The runtime reads logits at each marker, divides by the stored temperature and applies a softmax. The trade-off is accuracy for speed, since the 27B model scores higher in the base evaluation.

Build the joint prompt from your questions
Evaluate text and image tokens in one microbatch
Read the logits at each question marker
Divide by temperature and apply a softmax

Good to know: The scores below come from the base model, not these GGUF files.

Title: Overall Decision Index 0.3 on the base model
Flash, joint requests: 58.88 points
Flash, one question per request: 60.05 points
pplx-decider-v1.2, joint requests: 68.09 points

• Runtime: stock llama-server does not run protocol v3, so you need a compatible one.
• Precision test: on Apple Metal the Q8_0 file agreed with a float32 run on the top option for 13 test questions. The widest probability gap was 0.016.
• License: apache-2.0
• Cost to run: no API key, account or paid service named, though the base Python path needs a CUDA GPU.
• Data it reaches: files you download from Hugging Face, and the text and images you pass to the model.
• Not for: stock llama-server, or work where top accuracy matters more than speed and cost.

On Hugging Face, 13 November 2026

• Task: text classification
• Library: gguf
• Parameters: 4.8B
• Weights: about 2.4 GB at 4-bit (estimate)
• Formats: GGUF
• Downloads in the last 30 days: 11,185
• Likes: 10
• License: apache-2.0
• Based on: perplexity-ai/pplx-decider-flash-v1.2
• Tags: gguf, classification, multimodal, decider, llama.cpp, text-classification, conversational

Links

• Model: https://huggingface.co/perplexity-ai/pplx-decider-flash-gguf-v1.2
• Files: https://huggingface.co/perplexity-ai/pplx-decider-flash-gguf-v1.2/tree/main

Get the model

Download its files with Hugging Face's command line:

pip install -U huggingface_hub
hf download perplexity-ai/pplx-decider-flash-gguf-v1.2

Then follow the usage in its model card. Nothing in this kit runs by itself.